Compare commits

..
157 Commits
Author SHA1 Message Date
vhaudiquet 4b52f8e9e9 docs: add crates.io installation instructions
CI / build (push) Successful in 3m1s
CI / test (push) Skipped
CI / publish (push) Skipped
CI / snap (push) Successful in 6m26s
pkh v0.1.0 is published to crates.io, so cargo install pkh is now the
primary installation path; building from source stays as the
alternative. Drop the 'no distribution channel' notice.
2026-09-24 22:19:24 +02:00
vhaudiquet 6df490cf9a ci: install build-essential in the publish job
CI / build (push) Successful in 3m3s
CI / test (push) Skipped
CI / snap (push) Successful in 6m9s
CI / publish (push) Successful in 2m11s
cargo publish verifies the packaged tarball with a full build before
uploading, and that build needs a C linker for build-script crates
(proc-macro2 was the first to fail). The job's container setup step
omitted build-essential, unlike the build and test jobs, so the
verification failed with 'linker cc not found' on the v0.1.0 tag.
2026-09-24 21:41:15 +02:00
vhaudiquet f5b7704647 data: add an agent SKILL.md teaching pkh usage
CI / build (push) Successful in 3m1s
CI / test (push) Skipped
CI / snap (push) Successful in 6m1s
CI / publish (push) Failing after 53s
Add data/skill/SKILL.md in the agent-skills open format: a skill
directory that agents (Claude Code, Codex, OpenCode, ...) discover and
load on demand. It documents the shared option surface, the pull,
chlog, build, deb, lint, put workflow and the flags that keep runs
non-interactive; the command reference was generated from the live
--help output of every subcommand.

It lives under data/ so a later module can embed it and ship it with
the binary, for example a 'pkh skill' installer writing it into the
agent skill directories.
2026-09-22 23:55:56 +02:00
vhaudiquet 4c1edc7dcd docs: update example gif
CI / build (push) Successful in 3m13s
CI / test (push) Skipped
CI / publish (push) Skipped
CI / snap (push) Successful in 6m7s
2026-09-22 21:35:35 +02:00
vhaudiquet 9a66f8f7df deb: retry the interrupt chroot removal while children die off
dpkg defers SIGINT until it reaches a safe state, so it can still be
writing into the chroot when the watchdog's rm -rf starts racing
through it, failing with "directory not empty" and leaving the tree
half-removed. Retry the removal for a few seconds while the
interrupted children finish dying off; a genuinely stuck tree still
ends in the pkh prune message.
2026-09-22 19:57:01 +02:00
vhaudiquet 7a6337e1cb deb: build the tree the caller pointed at before the name search
CI / build (push) Successful in 3m16s
CI / test (push) Skipped
CI / publish (push) Skipped
CI / snap (push) Failing after 7m17s
build_binary_package stages the parent of the requested cwd, then
re-derived the package directory inside the staging area from
package/version name patterns plus the calling process's working
directory. That only works by accident for interactive users sitting
in the package directory: an embedded caller whose tree lives at
<job>/tree matches no pattern, and the process cwd means nothing to
a library consumer — the bc build above failed here even though the
tree was staged correctly.

The pointed-at tree is authoritative anyway: its changelog defined
the package, version and series for this build. Resolve its staged
copy outright when it carries a debian/ tree, keep the pattern
search (with the quirks overrides) as a fallback, and hand the
resolved directory to local::build instead of searching again.
2026-09-22 14:31:25 +02:00
vhaudiquet 9186bbbe51 deb: classify the staged listing through the context
The fallback listing of find_package_directory called Path::is_dir
on entries returned by list_files — a host-side stat. For an unshare
context (every local build) those paths are rooted inside the chroot
and do not exist at the same host path, so every entry came out a
non-directory, the 'Found directories' list silently stayed empty
and the failure degraded to the list-less 'Could not find package
directory' variant, hiding the actual layout (seen building bc from
an ubuntu/devel checkout whose staged tree was named 'tree').

Classify entries through the new ContextDriver::is_dir, and log
every entry instead of only those matching the broken host stat.
2026-09-22 14:31:10 +02:00
vhaudiquet 681fa3d687 context: add ContextDriver::is_dir
list_files returns context-relative paths (rooted inside the chroot
for an unshare context, on the remote for ssh): whether an entry is
a directory can only be decided through the context, never with a
host-side stat. Give every driver a proper is_dir rather than
approximating it with exists, so callers can keep directories and
files apart — the deb package-directory search lists directories
only.
2026-09-22 14:29:12 +02:00
vhaudiquet cc5bbd2297 chlog: drop a trailing buildN before appending ubuntu1
An Ubuntu upload of a package sitting at X-2build1 produced
X-2build1ubuntu1: the blind append misrepresents the lineage and,
sorting below the proper X-2ubuntu1, could never supersede it. A real
change on top of a rebuild replaces the marker instead, so the
trailing buildN is now stripped before the ubuntu counter is appended
or incremented: X-2build1 becomes X-2ubuntu1, X-2ubuntu1build1
becomes X-2ubuntu2.
2026-09-22 13:00:40 +02:00
vhaudiquet 5a1c1672cd cli: intercept Ctrl+C for deb, build and put
The SIGINT handler only records the interruption and wakes a watchdog
through a self-pipe (async-signal-safe); the watchdog runs the whole
shutdown in thread context — the live view's reporter first, then the
notice and the log hint, then the cleanup hooks, then exit 130. Flows
park in wait_for_shutdown instead of racing it with their own exit,
and an end-to-end test drives the sequence by re-spawning the test
binary and raising SIGINT at itself.
2026-09-22 10:44:05 +02:00
vhaudiquet adde0ee977 deb,ui: tear the live view down through the interrupt core
The ephemeral guard registers its chroot removal as an interrupt
cleanup hook and, once interrupted, stands down from its own teardown
so the two cannot race umount/rm; bootstrap bails out of tarball
extraction and the lockfile wait, keeping the hook registered on the
bootstrap error path so the watchdog can remove the partial tree. The
live view registers an interrupt reporter that suspends the widget and
returns the log-file hint, silences the tty rendering of the ^C
keypress (the echoed "^C" can wrap near the right edge and shift the
teardown erase by a row, leaving the first widget line on screen) and
kills the shared draw target so late log records cannot repaint the
cleared bars. Failure summaries stay quiet when interrupted: the
captured errors are just the killed children's death throes, and the
dose-builddebcheck diagnosis is skipped for a dependency failure the
user interrupted themselves.
2026-09-22 10:43:58 +02:00
vhaudiquet 84405a6762 interrupt: add the passive interrupt core
The library holds only the state its own types need when a Ctrl+C
arrives: the interrupted flag flows check to stand down, the cleanup
hook registry for resources that must not outlive the process (the
ephemeral build chroot), and the live view's reporter slot. The
signal handling itself is CLI wiring and lands separately: nothing
here installs handlers, prints or exits, so a library consumer
embedding these types keeps its own signal disposition.
2026-09-22 10:43:49 +02:00
vhaudiquet 8250a0e3b1 ci: publish the crate to crates.io on a v* tag
CI / build (push) Successful in 3m15s
CI / test (push) Skipped
CI / publish (push) Skipped
CI / snap (push) Failing after 7m34s
Trusted publishing is GitHub-Actions-only, so authentication goes
through a crates.io API token stored as the CARGO_REGISTRY_TOKEN
secret, scoped to the pkh crate. The job gates on the build job and
fails loudly when the tag does not match the version in Cargo.toml,
since cargo publish ships the declared version regardless of the
tag name.
2026-09-21 23:14:27 +02:00
vhaudiquet edfd7ed5ed snap: declare the dual license and ship the license texts
CI / build (push) Successful in 3m11s
CI / test (push) Skipped
CI / snap (push) Failing after 7m18s
The store listing carries the crate's MIT OR GPL-2.0-only
expression, and both texts ride along in the snap under
/usr/share/doc/pkh: the MIT grant requires the notice to
accompany copies, and GPL-2 requires the license text with
distribution.
2026-09-21 22:41:09 +02:00
vhaudiquet 6545d327e4 docs: add example gif
CI / build (push) Successful in 3m13s
CI / test (push) Skipped
CI / snap (push) Failing after 7m23s
2026-09-21 21:33:52 +02:00
vhaudiquet b118a54bff deps: describe the crate and dual-license it MIT or GPL-2.0-only
CI / build (push) Successful in 3m2s
CI / test (push) Skipped
CI / snap (push) Failing after 7m15s
Publishing to crates.io requires a description and a license; the
repository and readme give the crates.io page the right links. The
license texts ship as LICENSE-MIT and LICENSE-GPL.

Dual licensing is valid while no lintian-derived code is in tree:
shelling out to lintian is mere aggregation. When derived lint
collections are implemented they must land in a separate GPL-2-only
crate, since their copyright belongs to lintian's authors and
cannot carry an MIT grant; the core crate stays dual.
2026-09-21 20:46:46 +02:00
vhaudiquet 43f8a9e275 docs: remove roadmap, consolidate README
CI / build (push) Successful in 3m2s
CI / test (push) Skipped
CI / snap (push) Successful in 6m9s
2026-09-21 18:20:26 +02:00
vhaudiquet ff41edbd47 ui: create the rolling pane lazily, on its first line
DebUi seeded both widgets with a "(starting...)" placeholder,
replaced as soon as real content arrived during a build. pkh put
reports through the same view but runs no subprocess, so nothing
ever fed the pane: its seed line stayed on screen for the whole
upload, stacked under the per-file byte progress.

Drop the seeds and add the pane bar to the terminal only when the
first classified line arrives; a phase change (and the final
suspend) takes it off again. Flows without subprocess output now
render the status bar alone.
2026-09-21 15:20:17 +02:00
vhaudiquet 9e0b6a37a6 put: degrade to the anonymous FTP queue when the SSH transport fails
CI / test (push) Skipped
CI / build (push) Successful in 3m0s
CI / snap (push) Successful in 6m6s
pkh put only spoke SFTP to the PPA queue, so a failure of the SSH
transport itself (TCP, banner exchange) failed the upload even though
dput happily pushes the same files: its plain ppa: profile goes over
the anonymous FTP queue of ppa.launchpad.net, the same destination
over another port.

Classify the SSH connection failures: Transport (the connection never
came up: resolution, TCP, banner or key exchange) degrades to that FTP
queue — the upload order (payload first, .changes last), the
reverse-order DELE cleanup of a failed upload and the per-chunk
progress reporting all mirror the SFTP path, sharing cleanup_list.
Refused failures (host key not accepted, no matching authentication)
stay errors: silently switching transport would bypass the refusal.

The FTP client is suppaftp's blocking stream, with the time bounds it
does not carry by itself: the control channel's reads and writes, the
data channel's writes and connect (through a custom passive stream
builder), and the NAT workaround for PASV replies announcing an
unroutable address. The queue endpoints (host, port) join
data/launchpad.yml next to the SFTP ones, and the FTP transport is
covered by unit tests against an in-process fake queue plus a live
control-channel handshake with the real server (ignored, network).
2026-09-21 14:34:04 +02:00
vhaudiquet ac19fd9d65 deps: add suppaftp and parking_lot
suppaftp 12 is the FTP client behind the put FTP fallback transport:
the maintained continuation of rust-ftp (4.2M downloads, releases this
month), used with its default features only — a plain blocking FTP
stream, no TLS, no async. It brings just lazy-regex into the tree;
chrono is shared.

parking_lot replaces std sync mutex unwrapping in test code, per the
project rule.
2026-09-21 14:34:04 +02:00
vhaudiquet 02e1f739c3 chlog,cli,docs: offer suite-aliased series in the chlog selector
The chlog series selector only appeared when the changelog's current
distribution resolved to a known series; a Debian package targeting
'unstable' (or 'stable', 'testing', ...) fell through to keeping the
current series, silently, with no menu.

Resolve the changelog distribution through the suite aliases first
(unstable identifies the same series as sid, which resolves to the
Debian series list). The selector offers an aliased series as
'<suite> (<series>)' — 'unstable (sid)' — preselected, but selects
the suite name: what a changelog distribution field expects, instead
of the codename. Every other label and free-typed input selects
itself, unchanged.

Reflect the selector in the README roadmap checklist.
2026-09-21 11:52:39 +02:00
vhaudiquet 8c6f6f4028 distro_info: match changelog suite names with their series
Debian packages conventionally target 'unstable' in their
debian/changelog distribution field, but the series data (the
distro-info CSVs) only knows codenames: the suite is the alias
'unstable' of the series 'sid', a mapping the debian-distro-info tool
resolves internally without exposing it in its data.

Add a per-dist suite_aliases reference-data key (debian: unstable ->
sid), with two helpers on top: resolve_suite_alias, identifying a
changelog suite name with its series codename and the dist that
codename belongs to, and series_suite_alias, the inverse direction.
The two names identify the same series.
2026-09-21 11:52:39 +02:00
vhaudiquet 37e0b5c978 pull: fetch every component tarball of multi-orig packages
Sources listed with "3.0 (quilt)" extra components (node-jest, php-*,
...) carry one tarball per bundled module next to the main orig, named
<package>_<uver>.orig-<component>.tar.<ext>. fetch_orig_tarball picked
the single file matching ".orig.tar." — which cannot even match the
component naming — so a git pull only fetched the main orig. The later
dpkg-source -b quilt verification then failed with "can't find file to
patch" on the first patch touching a component directory.

Select the files with the existing build::changes::is_orig_tarball
helper (mirroring dpkg's \.orig(-.+)?\.tar\. strip pattern) and fetch
all of them, pristine-tar checkout first with a checksummed archive
download fallback, per tarball.

The end-to-end test now asserts every stanza-listed orig lands in the
package dir instead of just any *.orig.tar.* file, and gains a
node-jest (trixie, 24 components) regression case.

Verified live: pkh pull node-jest -d debian fetches all 15 origs of the
sid ds7 repack, and dpkg-source -b builds the debian.tar.xz and dsc
without touching the series.
2026-09-21 11:28:55 +02:00
vhaudiquet 6c0b200241 deps: commit the Cargo.lock
A binary crate should pin its dependency graph: without the lockfile
in git, source and snap builds float transitive versions, so a
0.1.0 artifact rebuilt later would not be the same binary.
2026-09-21 01:36:40 +02:00
vhaudiquet 4edf331444 docs: refresh command list, workflow example, roadmap and install
The command block is now the actual pkh --help output (new, lint and
prune were missing). The example workflow used pkh commit, a
subcommand that does not exist; commits go through git until
chlog/pkh commit land. The roadmap now reflects what is implemented
(pull -v, deb --mode local, lint, prune, new) and an installation
section documents the source build and its system dependencies.
2026-09-21 01:36:39 +02:00
vhaudiquet fa121f08ec cli: expose --version on the root command
The flag was disabled in the initial commit, leaving the binary with
no way to report its number — wrong for a release. clap scopes the
automatic version flag to the root command (-V/--version), so the
per-subcommand -v target-version options of pull and chlog are
unaffected.
2026-09-21 01:36:39 +02:00
vhaudiquet ac83a939e3 snap: confine classically and carry the full packaging toolchain
CI / build (push) Successful in 2m50s
CI / test (push) Skipped
CI / snap (push) Successful in 5m55s
A devmode snap is a smoke test, not a distribution channel: pkh
drives the whole host packaging stack (unshare chroots, overlay
mounts, dpkg/quilt/lintian across arbitrary paths), which only
classic confinement can express.

The snap now bundles every host-side tool pkh execs (git, gnupg,
dpkg-dev, quilt, pristine-tar, mmdebstrap, lintian, fakeroot,
util-linux, mount, schroot, openssh, tar/xz/bzip2), with apt and
dpkg deliberately left to the host: a core24 apt managing a newer
host's package database is exactly the skew classic snaps must
avoid. Tools running only inside the build chroot stay out; pkh
provisions those itself.

Release metadata comes from Cargo.toml instead of the git hash, and
grade is stable, so a build of any commit packs as the declared
version.

Classic-mode correctness: noble's mount/umount are staged from the
split mount package, fakeroot is exposed via symlink since
update-alternatives does not run at staging, and every bundled ELF
is patched to the core24 loader with a DT_RPATH resolving the base
and $ORIGIN. Without this the host loader would pin the snap to
hosts with a matching glibc, and the host ld.so.cache would mix
host libraries with base ones.
2026-09-21 01:22:10 +02:00
vhaudiquet 1145ca55eb docs: drop the pkh context roadmap items
The subcommand is gone from the CLI; context survives only as the
internal build backend, so there is no user-facing surface left to
track on the roadmap.
2026-09-21 00:10:45 +02:00
vhaudiquet b99f945b98 cli: remove the pkh context subcommand
The context management interface never worked reliably, and keeping
it exposed presents a feature that is not ready. Contexts remain in
the library as the execution backend for pkh deb (unshare chroots
and friends); only the CLI surface goes.
2026-09-21 00:10:45 +02:00
vhaudiquet ae5b0042e4 context: name temp dirs atomically, not probe-then-create
CI / build (push) Successful in 2m58s
CI / test (push) Skipped
CI / snap (push) Successful in 4m39s
create_temp_dir probed for a free pkh-<seconds> name and then created
the directory, so two contexts arriving together could both observe a
free name and unpack into the same directory — observed as two e2e
tests started within the same second failing on 'File exists when
hard linking' during the chroot tarball unpack.

Name with sub-second precision and create atomically: a losing race
gets AlreadyExists and falls through to the next attempt, which
removes the probe window instead of narrowing it. The schroot and
ssh drivers already use mktemp -d and need no change.
2026-09-20 19:47:24 +02:00
vhaudiquet 5b0cc08d8f deb,quirks: scope deb entries per series with an entry list
One deb block per package cannot express two series needing different
rules. Make pull and deb lists of entries, each with its own series
scope: every matching entry applies, in file order, so a stonking
entry can carry its own dependency rules next to the resolute one.
Thread the series through find_package_directory for the
package_directory lookup, which keeps its deb-then-pull fallback.
2026-09-20 19:40:19 +02:00
vhaudiquet ff1f8c7ccd deb,quirks: generic dependency rules for build-dep workarounds
CI / build (push) Successful in 3m3s
CI / test (push) Skipped
CI / snap (push) Successful in 4m38s
The resolute linux and linux-riscv controls declare llvm-21-dev
unqualified while their other llvm pieces are :native, so the dpkg
cross rules resolve it against the host architecture — whose
dependency closure needs python3:riscv64, conflicting with the
python3 the control itself declares :native. No resolver can install
that set; sbuild fails on it identically.

Give the deb quirks a dependencies rule set rather than a one-off
native-qualification knob: replace rewrites a declared dependency by
name with a full dependency string (qualifier, version and
restrictions included), inject adds dependencies resolved as if
declared, drop ignores declared ones. Entries are scoped by series so
they can be dropped when the upstream packaging catches up, and rule
names that match nothing are warned about so stale quirks surface.

Ship the llvm-21-dev entry for the resolute kernels: resolve it as
llvm-21-dev:native, keeping the declared <!stage1> restriction.
2026-09-20 19:10:06 +02:00
vhaudiquet 31fe6dc524 context: run commands in the C locale by default
Subprocesses inherit the session environment, so a translated host
locale leaked into builds: perl-based packaging tools
(dpkg-architecture, dpkg-parsechangelog, quilt, ...) warned about
missing locale settings, change their output with the environment,
and dpkg-buildpackage treats some of that output as data.

Default every command to LANG=C and LC_ALL=C; a caller can still
override explicitly through envs().
2026-09-20 19:06:04 +02:00
vhaudiquet 5592d5a6e4 test: cross-build linux-riscv from resolute, noble known-broken
The e2e pulled linux-riscv from questing, which went EOL in July
2026 and has since been removed from the mirrors entirely — not even
old-releases carries a Release file for it, so the chroot bootstrap
cannot resolve the series and the e2e can no longer run.

Point it at resolute (26.04), the current LTS: linux-riscv is
packaged there like in every supported series, and an LTS stays
pullable for years where an interim series turns the test stale
within months.

Keep a noble fixture next to it, also not run by default: noble-era
controls declare their build tools unqualified (the :native idiom
landed in later series), so exact dpkg semantics demand
host-architecture instances of them (python3:riscv64, gcc-13:riscv64,
...), and the two-architecture install set is unsolvable — t64
libraries conflict with their own foreign-arch variant, and the
riscv64 toolchain instances drag depends chains that do not resolve
from the chroot sources.
2026-09-20 19:05:59 +02:00
vhaudiquet bb719e7c80 apt/keyring: keep the keyring cache readable
A fresh chroot download failed with mmdebstrap unable to copy the
keyrings: its unshare-mode hooks run under an identity that cannot
read the invoking user's private directories, so the 0700 cache
directory the module created broke the trusted.gpg.d setup hook with
'Permission denied' (builds reusing an already-cached tarball never
hit the path).

Create the cache 0755 and chmod the keyrings 0644 instead. The
planting guard does not weaken: cached keyrings are trusted as-is,
and validate_keyring_dir keeps refusing directories that are not
owned by the current user or are writable by group or others — read
access for the bootstrap tool is not a planting vector. Legacy 0700
cache directories are validated as before, then widened.
2026-09-20 18:14:28 +02:00
vhaudiquet af9845c480 deb: install build-deps per dpkg cross semantics
Replace the apt-get build-dep passes with a native resolver: the
Build-* clauses are evaluated against the context's package state by
crate::debian::deps (the dpkg-checkbuilddeps equivalent), and the
unsatisfied ones install through explicitly architecture-qualified
names.

apt's --host-architecture build-dep resolution is coarser than the
checker it feeds: unqualified Multi-Arch: same libraries only ever
get the host-architecture variant, and there is no way to also
satisfy the build-architecture needs of a cross build without a
native re-resolution pass — which installs co-install partners that
break unpacking for -dev packages whose variants conflict on
arch-differing files (curl-config; the libcurl4-gnutls-dev
regression). The resolver now applies the same dpkg rules as the
checker, preferring the runnable build-architecture variant of
Multi-Arch: foreign tools, and passes virtual names through to apt.

Build-Conflicts are now checked before anything installs, which the
build-dep passes never did, and a failed dose3 diagnosis no longer
masks the resolver error (a latent flaw of the passes: binary-only
builds have no .dsc for dose-builddebcheck to read).

An end-to-end test pins the expected failure: a cross build whose
build-dependencies cannot satisfy dpkg's cross semantics aborts,
naming the dependency.
2026-09-20 18:04:13 +02:00
vhaudiquet 02cb5306f3 debian: test the cross build-dep lookup against dpkg
The Facts lookup already implemented Dpkg::Deps::KnownFacts
_find_package, but nothing pinned its cross-compilation behaviour:
with host != build, an unqualified dependency resolves against the
HOST architecture instance (or any instance of a Multi-Arch: foreign
package, or an Architecture: all one) and never against the
build-architecture instance of a Multi-Arch: no/same package.

Cover the matrix with unit tests and, like the source-build checker,
with a differential sweep against real dpkg-checkbuilddeps on a
synthetic admindir with -a <host>: the checker and the real tool
agree on every fixture, including the quirks (:native aborting on
Multi-Arch: foreign instances, first-match version binding).
2026-09-20 18:04:05 +02:00
vhaudiquet 5c026a7050 docs: add AGENTS.md with the tree conventions
CI / build (push) Successful in 3m12s
CI / test (push) Skipped
CI / snap (push) Successful in 4m32s
Codify what the history already does so agents and new contributors do
not have to reverse-engineer it: the fmt/clippy gates mirroring CI's
-Dwarnings, the <scope>: <summary> commit format with the per-module
scope table, and the library-side invariants (missing_docs, report
ports, data/*.yml embeds, README parity for user-facing changes).
2026-09-20 16:30:15 +02:00
vhaudiquet 9238aa961f chlog: fall back to the changelog history when no version tag exists
CI / build (push) Successful in 2m56s
CI / test (push) Skipped
CI / snap (push) Successful in 4m30s
get_commits_since_version silently returned an empty change list when
the previous version carried no tag. Walk the history back to the last
commit that modified the changelog itself and use it as the boundary
instead, so entries stay correct in repositories that commit their
changelogs without tagging them.

Tag detection remains the preferred path. Before falling back to the
changelog commit, the version recorded by the committed changelog is
probed for a tag: an uncommitted newer entry on top (e.g. UNRELEASED
from a previous run) does not hide the previous version's tag.
2026-09-20 00:38:15 +02:00
vhaudiquet d1056fbbbf ui: ellipsize fake-terminal pane lines wider than the terminal
Overflowing log lines in the pkh deb / pkh build rolling pane used to
wrap onto a second line, corrupting the pane layout. Truncate them to
the terminal width (minus the pane prefix) by display width and append
an ellipsis instead; lines are left whole when the terminal size is
unknown.
2026-09-20 00:15:37 +02:00
vhaudiquet a0e74073bf lint: add pkh lint, wrapping lintian for parity plus pkh-native checks
CI / build (push) Successful in 2m55s
CI / test (push) Skipped
CI / snap (push) Successful in 4m32s
pkh covered the package lifecycle but never validated the packaging
itself: broken control stanzas, unparsable changelog versions or
uncommitted debian/ edits only surfaced at build or upload time. pkh
lint lints a source tree with day-one lintian parity plus a native Rust
engine for the checks lintian cannot have.

The wrapper reuses the pkh build output next to the tree when it matches
the current changelog entry and no tree content is newer (mtime walk,
skipping .git/.pc), else packs fresh with dpkg-source -b using weak gzip
compression (the artifact is ephemeral; xz dominated the run at 9.8 s
versus 2.7 s on a 111 MB tree) and symlinks quilt orig tarballs from the
tree's parent, which dpkg-source searches in cwd. Findings are parsed
from the installed lintian into a unified report, deduplicated by tag
name against the native engine, and rendered lintian-shaped
(<L>: <pkg> <type>: <tag> <details>) as text or JSON, colorized at
render time (--color auto/always/never). Exit codes follow lintian's
contract (0 clean, 1 findings at/above --fail-on, 2 runtime error);
lintian's own exit code is ignored because it uses 2 both for findings
and for runtime errors. -d/--dist maps to lintian --profile so the
target distro's rules apply even on a foreign host.

The native engine hosts the first workflow check lintian cannot know:
pkh-debian-changes-not-committed flags debian/ content that is not
committed to git, since the pkh flow builds and uploads the tree as-is.
Checks register in a static registry validated by a unit test, and the
wrapper's parser is pinned by golden tests captured from lintian 2.129
output. Strategies for lintian's Ubuntu blind spots (its vendor data
there is one file plus 14 disabled tags) are specced in
plans/pkh-lint.md, deliberately not implemented yet.
2026-09-19 23:45:31 +02:00
vhaudiquet 4f5246ccd3 chlog: number Ubuntu backports with the per-release SRU scheme
CI / build (push) Successful in 2m56s
CI / test (push) Skipped
CI / snap (push) Successful in 4m35s
--backport was Debian-only: ~bpo is backports.debian.org's scheme and
its number the Debian release, so Ubuntu targets were rejected outright.
Ubuntu backports have their own documented scheme (Ubuntu version-
strings): the development release's version with a per-release ~YY.MM.1
appended, sorting before it (3.1-1ubuntu2 backported to 22.04 becomes
3.1-1ubuntu2~22.04.1; native 3.1 becomes 3.1~22.04.1) and independent of
the version the target release carries. The .N increments for
subsequent per-release SRU uploads.

backport_series_number becomes backport_suffix_for_series: the release
number comes from the new generic get_series_release_number (version
column of the target series' own distro-info data, leading token kept —
"12" for bookworm, "26.04" out of resolute's "26.04 LTS"; empty column
as on sid/experimental means None), and the suffix is picked per vendor:
~bpoNN+ for Debian (plain integer releases only), ~YY.MM. for Ubuntu.
Unnumbered series still error before anything is written.

Also serialize the changelog tests that mutate the process-global
DEBFULLNAME/DEBEMAIL variables behind a tokio Mutex: run in parallel
they raced each other's identity reads, which started failing
intermittently as generate_entry tests accumulated.
2026-09-19 21:32:02 +02:00
vhaudiquet dd2438a72c chlog: number regular uploads after the target series' vendor
A flagless entry (no --backport/--nmu/--rebuild) targeting an Ubuntu
series was numbered the Debian way: 1.0-1 became 1.0-2 with distribution
noble, and getting the conventional 1.0-1ubuntu1 required hand-editing
the version. The bump now derives from the vendor of the target series
(distro-info): Ubuntu series get the ubuntu suffix convention (1.0-1
becomes 1.0-1ubuntu1, and re-bumping an already-Ubuntu changelog
increments the counter instead of the revision), Debian series keep the
plain revision bump, and series that cannot be resolved to a vendor
(UNRELEASED, unknown) fall back to it too. There is no reverse sync, so
the Ubuntu-to-Debian direction needs no special casing.

Runs with an explicit --series now consult distro-info once, where they
previously queried it not at all; the interactive flow already did for
the series selector. EntryKind::Ubuntu remains the library-level way to
force the numbering regardless of the series.
2026-09-19 20:58:44 +02:00
vhaudiquet 012df20961 chlog: number entries as backport, NMU or no-change rebuild
The version flags were never reachable: --backport was declared but not
read, and compute_new_version's NMU/rebuild numbering sat behind a TODO
asking for CLI wiring (the old positional-bool signature always received
false). generate_entry now takes an EntryKind selected by three mutually
exclusive flags, and is async because the backport numbering derives the
Debian release number of the target series from distro-info:

- --backport: 1.0-1 becomes 1.0-1~bpo12+1 (12 = release number of the
  target series, backport suite names accepted too). Re-running on an
  already-numbered version bumps the counter; series without a numeric
  Debian release (sid, Ubuntu series, UNRELEASED) are rejected before
  anything is written.
- --nmu: 1.0-1 becomes 1.0-1.1 (native 1.0 becomes 1.0+nmu1).
- --rebuild: 1.0-1 becomes 1.0-1build1.

An explicit --version overrides all three. compute_new_version went
from four positional bools to a private Bump enum; backport numbering
reuses increment_suffix with a '~bpoNN+' suffix. The CLI prints the
computed new version before opening the editor.
2026-09-19 20:56:03 +02:00
vhaudiquet 47bb7c608e deb: resolve cross pkg-config against the target multiarch
CI / build (push) Successful in 2m54s
CI / test (push) Skipped
CI / snap (push) Successful in 4m32s
In-tree tools locate their libraries with the *host* pkg-config during
cross builds (the kernel's tools/build feature checks derive their
cflags/ldflags from 'pkg-config --cflags/--libs'), whose search path
only covers the build architecture's pkgconfig dirs. Cross builds of
linux-riscv died in rtla's Makefile.config: libtraceevent/libtracefs
were reported missing although the riscv64 -dev packages were
installed, because pkg-config never saw their .pc files.

The host-arch -dev packages that used to make those checks pass came
from the unscoped arch-indep build-dep pass, whose native
re-resolution b34e86d correctly scoped to the host arch — removing the
accidental co-install along with the bug it papered over. Export
PKG_CONFIG_LIBDIR for the target multiarch instead, so the checks
resolve target-arch libraries directly: no native build-dep bloat and
no wrong-arch linking.
2026-09-19 13:07:46 +02:00
vhaudiquet 6caedce61a report: reproduce the pre-refactor CLI output through the ports
Instead of carrying raw UI in core, the ports now represent everything
the CLI used to do inline:

- Prompter::present shows context outside of a question (the wizard
  summary screen, the vendoring notice spacing); TerminalPrompter
  prints it on stdout exactly like the println!s it replaces, server
  embeds forward it as a display event.
- generate_entry returns the generated entry (package, versions,
  series, path) instead of printing; the CLI renders the same lines.
- BuildTarget carries a flow-composed display line and a tee_log flag:
  the terminal adapter renders it verbatim ("Building source package
  ...", "Building ... for series/arch", "Uploading ... to ...") and
  uploads open no build log.
- The unmet build-dependency diagnostics are rendered by the CLI from
  the typed error, in the original order (details, then summary).
- --verbose constructs no live view at all (an idle widget used to
  linger), and the re-vendor offer only logs when it is actually
  asked, so headless runs print the error exactly once.
2026-09-19 00:52:31 +02:00
vhaudiquet bd8f814a53 Revert "package_info: drop the ANSI color from the not-found warning"
This reverts commit 4fae02bc85.
2026-09-19 00:39:29 +02:00
vhaudiquet 4fae02bc85 package_info: drop the ANSI color from the not-found warning
The location embedded in the log record was styled with crossterm: a
remote consumer of pkh's log records would receive ANSI codes inside
the message text. Plain text is the logger's business to style.
2026-09-18 20:55:56 +02:00
vhaudiquet c2dae4f3f9 lift main.rs business logic into the library
The chlog target-series resolution becomes changelog::series_candidates
(UNRELEASED pinning, development-series default and fallbacks modeled
by SeriesCandidates), PPA references get package_info::split_ppa
(shared by pull and deb, now also rejecting empty parts), and SSH
endpoints get context::ContextConfig::from_endpoint — so a library
consumer can resolve series, validate PPAs and build context
configurations without reimplementing the CLI's rules. All three carry
unit tests; main.rs keeps only parsing of flags and error handling.
2026-09-18 20:51:36 +02:00
vhaudiquet 0421a91e01 changelog: report through the log instead of printing to stdout
generate_entry's status messages become log lines, and the unmet
build-dependency diagnostics travel inside the UnmetBuildDependencies
error (its Display carries the full report) instead of being printed
to stderr by the library: the caller renders both like any other
outcome.
2026-09-18 20:44:05 +02:00
vhaudiquet 54cb04ba27 put: report through the view and ask the host-key question through the Prompter
put() loses its MultiProgress parameter: the summary, pre-flight and
connection spinners become view messages, the per-file SFTP transfer
reports determinate progress through view.progress (upload_file takes
a byte-count callback instead of an indicatif bar), and the display is
released through view.suspend on every exit path. The hardcoded
trust-on-first-use prompt in the SSH host-key verification becomes the
Prompter::accept_host_key port (fail-closed by default; the terminal
prompter prints the authenticity banner and confirms), so a remote
frontend can surface its own host-key dialog.
2026-09-18 20:42:17 +02:00
vhaudiquet bb76e41908 new: drive the wizard and verification offers through the Prompter port
The interactive half of pkh new no longer touches the terminal prompt
module directly: run() takes a Prompter, picks the wizard or the plain
resolve path through interactive(), and every select/text/confirm
question (including the verification offers) goes through the port.
Cancellations propagate as Err, preserving Ctrl+C-aborts; the summary
and vendoring-notice prints become log lines. A builder-server embed
can now drive the whole scaffold wizard over its own wire format by
implementing Prompter.
2026-09-18 20:38:05 +02:00
vhaudiquet d64e472845 report: grow the Prompter port and move display_path out of the ui module
Prompter gains interactive(), select() and text() (with the Validator
type), and confirm() now propagates cancellation as Err so flows abort
instead of silently taking a default when the user hits Ctrl+C. The
terminal prompter implements the full port; the port also re-exports
the path display helper, which is pure presentation formatting used by
events and messages rather than terminal code.
2026-09-18 20:34:33 +02:00
vhaudiquet 052c02cdc3 deb: drive build_binary_package through the BuildView port
The binary build joins the source build on the reporting ports:
build_binary_package takes a DebBuildOptions struct (replacing eleven
positional arguments), reports target, phases, progress and the
outcome through the environment-agnostic BuildView, and the Phase enum
with its default classifiers moves from the terminal widget into the
deb module (announced through the enter_phase helper). DebUi loses its
inherent event methods and only implements the port; tee logging and
the SIGINT behavior are unchanged.

No behavior change for the CLI; headless consumers pass report::Quiet.
2026-09-18 20:22:45 +02:00
vhaudiquet 27b1083b15 report: add BuildView/Prompter ports and drive pkh build through them
Core flows no longer reach into the terminal UI: build_source_package
takes a BuildSourceOptions struct (source tree, domain options, view,
prompter) and reports phases, messages and outcomes through the
environment-agnostic ports in the new report module. The classifiers
move from ui/logfmt to the core logfmt module, DebUi becomes a
BuildView adapter, the re-vendor retry asks the prompter instead of
checking for a TTY, and artifact/success printing moves to the CLI.

Headless consumers pass report::Quiet; an embedding (e.g. a builder
server forwarding events to a web frontend) implements BuildView and
maps the plain-data events onto its own wire format.
2026-09-18 20:02:56 +02:00
vhaudiquet a7d2cfdc6e ci: pin upload-artifact to v3
CI / build (push) Successful in 2m57s
CI / test (push) Skipped
CI / snap (push) Successful in 4m29s
v4's GHES check refuses to run on any non-github.com server, so the
snap artifact upload always failed on gitea; v3 uses the artifact API
gitea implements.
2026-09-18 18:07:54 +02:00
vhaudiquet ff635b305b ci: install nodejs in the snap job so JS-based actions can run
CI / build (push) Successful in 2m55s
CI / test (push) Skipped
CI / snap (push) Failing after 4m19s
2026-09-18 16:33:16 +02:00
vhaudiquet 781ed204c2 clippy: write the changelog body loop as while let (rust 1.98)
CI / build (push) Successful in 2m53s
CI / test (push) Skipped
CI / snap (push) Failing after 9s
2026-09-18 15:43:12 +02:00
vhaudiquet 0235ec6457 new: port the python template bodies to manifests
CI / build (push) Failing after 2m55s
CI / test (push) Skipped
CI / snap (push) Skipped
2026-09-18 15:20:22 +02:00
vhaudiquet fa399f64b2 new: port the meson, cmake and autotools template bodies to manifests 2026-09-18 15:08:27 +02:00
vhaudiquet e6f2012835 new: port the rust template bodies to manifests 2026-09-18 15:05:44 +02:00
vhaudiquet ffac4d6b57 new: port the shell, empty, makefile and go templates to manifests
Move the four templates' static file bodies into .tpl files under
data/templates/<id>/, referenced by their manifests' files: lists —
the shell skeleton script (executable, {command}-named) with its
skeleton-only debian/install mapping, the empty template's stub README,
the makefile hello.c/Makefile skeleton with its skeleton-only install
mapping, and go's go.mod/main.go skeleton (the go directive of go.mod
stays a literal: nothing about it is answer-derived).

The empty template ends up hookless — zero Rust, its registry entry
points at no hooks — and src/new/templates/empty.rs is deleted. The
shell and go hooks shrink to their probes (plus go's {go_import_path}
context value); the makefile hooks keep only the existing-tree hint
probing the packaged Makefile for a phony install: target, since that
heuristic reads the tree and cannot be data.
2026-09-18 14:58:13 +02:00
vhaudiquet 04a572cd77 new: template manifests and registry infrastructure
Split the Template trait into a data half and a logic half. Every
template is now declared by a manifest under data/templates/<id>/
(CLI id, wizard label, detection markers, Build-Depends, architecture,
rules dh line, rules-extra body, control source fields, gitignore
entries and static file bodies with {placeholder} substitution),
embedded through the TEMPLATE_SOURCES index and parsed once into the
registry; the order of the index is the wizard menu order and the
detection priority at once. The logic half is the slim TemplateHooks
trait (probe, post_write, file-body overrides merged over the manifest
bodies by path shadowing, Build-Depends/architecture amendments and
extra context values), registered per template as a HOOKS static: a
template without hooks needs zero Rust.

- TemplateId becomes a Copy wrapper of the stable CLI string; the
  enum, its all/as_str/display_name/from_label matches and the old
  statics array collapse into the registry accessors.
- rust's rules overrides move to data/templates/rust/rules.extra.tpl
  with {locked}/{artifact} hook context; python's backend table,
  meson/cmake's pkg-config opt-in, autotools' gettext and python's
  C-extension hints become hook amendments over the manifest baseline.
- detect.rs drops its hardcoded marker cascade: the manifests'
  detect.files drive detection in registry order, with the shell
  single-script heuristic and the never-detected empty template kept
  as the code special cases they are. License sniffing is untouched.
- The template tests port to manifest validation: registry coverage
  and stable order, placeholder presence in the rendering context,
  rules composition, the Build-Depends/architecture/dh-line table now
  asserted against the manifest data, and the hook shadowing merge.

The static skeleton bodies of the shell/empty/makefile/go templates
stay in their Rust hooks for now; the next commit moves them into
their manifests.
2026-09-18 14:46:28 +02:00
vhaudiquet fc0d2f247e build: resolve the default build profiles from the distro data 2026-09-18 13:58:57 +02:00
vhaudiquet ccd2e37385 deb: drive the cross-build repository setup from the distro data 2026-09-18 13:58:27 +02:00
vhaudiquet 52d5ad064f deb: match archive sources and the universe gate through the distro mirror data 2026-09-18 13:57:22 +02:00
vhaudiquet b8b2be5acf distro_info: model distro mirrors, components and build profiles in the yaml data 2026-09-18 13:56:36 +02:00
vhaudiquet dd1c70c91a new: drive the license menu, parsing and sniffing from data/licenses.yml 2026-09-18 13:52:24 +02:00
vhaudiquet 12407c8eac new: drive the forge hosts and tarball templates from data/forges.yml 2026-09-18 13:30:42 +02:00
vhaudiquet 7601524b7c apt: move the keyserver lookup URL to data/keyserver.yml 2026-09-18 13:28:05 +02:00
vhaudiquet 7af767898e launchpad: drive the endpoints from data/launchpad.yml 2026-09-18 13:27:20 +02:00
vhaudiquet ead97e1213 data: consolidate the YAML embed convention into an embed_data! macro 2026-09-18 13:22:13 +02:00
vhaudiquet 6e5ecd2f45 data: move the YAML data files into a top-level data/ directory 2026-09-18 13:15:29 +02:00
vhaudiquet 69f3c3954e new: build the target-distribution menu from supported_dists 2026-09-18 13:10:46 +02:00
vhaudiquet c106ebe315 apt: query the Launchpad PPA API through the shared HTTP client 2026-09-18 13:09:46 +02:00
vhaudiquet 941f91decf distro_info: read the YAML local series path, drop the dead dist_info stanza 2026-09-18 13:09:02 +02:00
vhaudiquet b489db2728 quirks: handle packages without quirks in deb extra dependencies 2026-09-18 13:08:23 +02:00
vhaudiquet c1f8893576 put: run the section pre-flight against the uploaded tree
CI / build (push) Failing after 2m57s
CI / test (push) Skipped
CI / snap (push) Skipped
The Section check always read debian/control from the current working
directory, so 'pkh put --changes ../other/pkg_changes' validated the
wrong tree. With an explicit --changes the check now runs against that
file's own directory when it holds debian/control, and is skipped with
a warning otherwise; tree uploads are unchanged.
2026-09-18 10:40:48 +02:00
vhaudiquet e640b153bd put: remove partial uploads when the transfer fails
A failed or interrupted upload left the already-uploaded payloads — or
a truncated .changes — in the PPA's incoming area. On failure the
already-uploaded files are now removed best-effort in reverse upload
order with the failed file first, so a .changes never outlives the
payloads it references; the original upload error keeps precedence over
cleanup failures, and record-after-success semantics are unchanged (a
failed upload must not count as uploaded).
2026-09-18 10:36:01 +02:00
vhaudiquet 231c478d0b put: surface known_hosts problems instead of silently degrading
An unreadable or unparsable known_hosts file was swallowed with
'let _', silently downgrading to prompt-and-accept without telling the
user why their configuration was ignored: warn naming the file, then
continue. And when the pinned Launchpad fingerprint matches, a
DIFFERENT key recorded for that host in known_hosts was silently
bypassed: warn about the stale entry (diagnostic only — the published
fingerprint stays authoritative).
2026-09-18 10:28:34 +02:00
vhaudiquet 1aa0ca3d2f clippy: collapse nested ifs into match guards (rust 1.98) 2026-09-18 10:19:16 +02:00
vhaudiquet b20acf3199 put: bound the SSH connect and session operations with timeouts
TcpStream::connect and the blocking libssh2 session had no timeouts: a
black-holed host hung pkh put forever, mid-resolution, mid-handshake or
mid-upload. Connect attempts now get a 15 s timeout per resolved
address, the session gets a 30 s API timeout for the handshake/auth
phase and a 300 s per-call timeout for SFTP operations (per low-level
libssh2 call, not per transfer — documented); failures name the
operation and host.
2026-09-18 02:03:20 +02:00
vhaudiquet 3feca504fc put: back up a corrupt uploads.json and write it atomically
A corrupt upload log was silently treated as 'never uploaded',
disabling the duplicate-upload guard without a diagnostic, and
record_upload truncated the file in place — a crash mid-write produced
exactly that corrupt state. Parse failures now log an error, back the
file up to uploads.json.bak (so a later successful upload cannot
destroy the recoverable history) and continue with an empty log; the
log itself is written to a temp file and renamed into place.
2026-09-18 01:53:52 +02:00
vhaudiquet 85f0d7d92f put: paginate the Launchpad published-sources lookup
The superseded check read only the first getPublishedSources page
(Launchpad defaults to 75 entries per page), so a source with a long
publication history could hide its true maximum published version and
let a superseded upload through, only to be rejected by the queue
hours later. Follow next_collection_link (ws.size=100, hard cap of 20
pages beyond which the check errors rather than risk a false 'not
superseded').
2026-09-18 01:49:50 +02:00
vhaudiquet c45edcee76 build: refuse source build of a binary-only changelog entry
dpkg-source errors with 'building source for a binary-only release'
when asked to -b a tree whose newest changelog entry sets
binary-only=yes: the source publication is already in the archive and
is not being rebuilt. pkh instead built the fresh .dsc and then
produced binNMU-style metadata referencing the *previous* version's
.dsc and tarballs — behavior dpkg does not have at all.

Mirror dpkg: run_source_build now refuses binary-only entries outright,
which makes the previous-version references, the binNMU Source field
and the Binary-Only-Changes handling in the source pipeline dead code —
removed. Binary-only metadata stays in the binary pipeline, where it
matches dpkg-genchanges/genbuildinfo (diff_binmu_binary_metadata).

New tests: a unit test for the refusal, and a failure-parity
differential asserting both dpkg-buildpackage -S and the native
pipeline reject the same fixture.
2026-09-18 01:13:19 +02:00
vhaudiquet 1fc1d1aa77 build: render the source .buildinfo from its own checksum set
The source pipeline used one checksum map for two documents with
different content: the .buildinfo (which, like dpkg-genbuildinfo, lists
only the referenced .dsc) and the .changes (which distributes the dsc,
the tarballs and the buildinfo itself). Because the tarballs and the
buildinfo were inserted into the shared map before the signing cascade
re-rendered the .buildinfo, every signed source build produced a
.buildinfo listing the tarballs — which dpkg-genbuildinfo never emits —
and itself, with the stale digest of its own pre-signature content.

Split the map: the .buildinfo renders from the referenced .dsc only
(refreshed after the .dsc is signed), the .changes keeps the full
distribution set with the signed buildinfo's fresh digests. Verified
with a throwaway GPG key: the signed .buildinfo lists exactly one entry,
the .dsc, matching the signed file.
2026-09-18 00:25:17 +02:00
vhaudiquet af870cb7cb build: stop redistributing the previous source on binNMU uploads
dpkg-genchanges/genbuildinfo handle a binary-only upload by referencing
the previous source version textually (Source: pkg (prev),
Binary-Only: yes, Binary-Only-Changes) while distributing no source
files at all: pkh instead pulled the previous .dsc and its tarballs
into both documents whenever they sat next to the artifacts, re-uploading
the whole source on every binNMU.

Drop that redistribution (and include_dsc_artifacts with it), and emit
the missing Binary-Only: yes field, which the new differential test
against real dpkg-buildpackage -b caught. The binNMU case shares its
runner with the regular binary metadata differential; a unit test pins
the exclusion even with the previous artifacts present.
2026-09-18 00:24:42 +02:00
vhaudiquet 2b017dcf43 new: keep vendored *.orig files through dh_clean in the rust rules
CI / build (push) Failing after 2m58s
CI / test (push) Skipped
CI / snap (push) Skipped
dh_clean unlinks *.orig patch backups, and vendored crates carry
Cargo.toml.orig (and the occasional *.xml.orig) that cargo's per-file
checksums require on cold builds. Override dh_clean with -X .orig.
2026-09-17 23:54:52 +02:00
vhaudiquet 4ab41e691a build: only redistribute the orig tarball on new upstream (-si)
dpkg-genchanges includes the upstream tarballs in the .changes only when
the upload brings a new upstream: no previous changelog entry, a changed
upstream version or a renamed source. On a plain revision bump the
tarball already sits in the archive, and dpkg strips it (and its .asc)
from the distribution set.

pkh's native source pipeline listed every .dsc-referenced tarball
unconditionally, making every upload re-ship the orig. Implement the
dpkg source styles as --orig auto|always|never (auto being the -si
default; always/never are -sa/-sd), stripping the tarballs out of the
changes, buildinfo-free checksum set and artifact list like dpkg, with
the explicit 'never' ignored for native packages. Comparison uses the
epoch-less upstream version, exactly like dpkg's version().

Differential tests against real dpkg cover revision bumps, new upstream
versions and both forced styles.
2026-09-17 23:45:15 +02:00
vhaudiquet c18f1fe9c2 changelog: parse entries with a shared limit-based helper
Replace parse_previous_version/parse_previous_version_from_str with
parse_changelog_entries(path, limit: Option<usize>), parsing up to the
given number of entries (None: the whole file) newest-first through the
same strict entry parser instead of a header-only scan. The single-entry
helpers stay as thin wrappers, and callers needing the previous entry
now get its full source name and version, not just the raw string.
2026-09-17 23:34:42 +02:00
vhaudiquet e7b35f5c37 deps: treat corrupt Provides as undecidable, not unmet
CI / build (push) Failing after 2m58s
CI / test (push) Skipped
CI / snap (push) Skipped
A versioned Provides whose version failed to parse was silently
skipped, so a corrupt dpkg status entry could yield a wrong 'unmet'
verdict where the truth is 'cannot decide': unparseable provided
versions now set lackinfos like unparseable installed versions do.
Provides alternatives with a non-= constraint are likewise rejected as
a whole field (dpkg rejects the entry), replacing the skip-per-
alternative behavior that contradicted the code's own comment.
2026-09-17 20:08:20 +02:00
vhaudiquet 54274e9079 version: reject an empty debian revision
DebianVersion::parse accepted '1.0-' (empty revision after rsplit on
the last hyphen), where dpkg rejects it with 'revision number is
empty'; downstream filename construction produced garbage like
'foo_1.0-.dsc'. Keep the start-digit warning-only semantics of dpkg
(no new check there) and the accepted '1.0--1' split.
2026-09-17 19:52:32 +02:00
vhaudiquet 93296c26c1 checksums: add SHA-512 and a Checksums-* field parser
The checksum model only carried md5/sha1/sha256 while deb-buildinfo(5)
defines Checksums-Sha512, and there was no way to parse a Checksums-*
field body back into entries. Add ChecksumKind::Sha512 (computed
alongside the others), a field parser validating the
'<hex> <size> <name>' grammar, and an only-if-populated
Checksums-Sha512 emission in .buildinfo — deliberately dormant in the
dpkg-parity flows, which never emit it, and .dsc/.changes untouched.
2026-09-17 19:44:54 +02:00
vhaudiquet d6bad9fbbe debian: make Paragraph::set drop case-insensitive duplicates
set only replaced the first match and appended otherwise, so a
paragraph holding both 'Depends:' and 'depends:' kept a stale second
value after an update, silently re-emitted on serialization. set now
updates the first match in place and removes any other case-insensitive
duplicate; the parser stays lenient and keeps duplicates reachable via
iter().
2026-09-17 19:16:28 +02:00
vhaudiquet 607711a6b5 build: surface a panicked output-reader thread
run_command_capturing discarded the pump threads' join results: a
reader that died mid-capture (UI sink or log writer failing) reported a
successful build with truncated captured logs. A reader panic now fails
the command; when the child itself failed first, its error keeps
precedence and the reader panic is logged so the truncated output is
not silently lost.
2026-09-17 19:12:35 +02:00
vhaudiquet 47b462ad61 build: merge inherited DEB_BUILD_OPTIONS instead of overwriting it
The source-build pipeline exported its computed DEB_BUILD_OPTIONS
verbatim, silently dropping options the user set in the environment
(e.g. terse) where dpkg-buildpackage prepends the inherited value.
Options are now merged inherited-first through a shared helper, with
whitespace normalized.
2026-09-17 19:05:15 +02:00
vhaudiquet fa39851783 build: check stat's exit status and honor DPKG_ORIGINS_DIR
hashes_in_context never checked stat's exit status and parsed its size
with unwrap_or(0), silently recording zero-size artifacts in the
generated .changes/.buildinfo; stat failures and unparsable sizes are
now errors naming the file. current_vendor hardcoded
/etc/dpkg/origins/default while dpkg honors DPKG_ORIGINS_DIR (already
in the file's own ENV_ALLOWED list); the origins default is now
resolved against it with the usual fallback.
2026-09-17 18:56:32 +02:00
vhaudiquet 22e43741f3 build: make build_source_package(None) default to the current directory
None mapped to Path::new("."), whose parent is the empty string:
the output-directory derivation then always failed with 'cannot
determine output directory', making the documented Option default a
guaranteed-failure trap. Resolve None to the process's absolute
current working directory instead.
2026-09-17 18:50:09 +02:00
vhaudiquet 36875513ee build: stop turning read failures into silently wrong metadata
Two read_file(...).unwrap_or_default() calls masqueraded IO errors as
empty data: an unreadable debian/files became 'binary build with no
binary artifacts found; cannot distribute', and an unreadable dpkg
status file produced an empty Installed-Build-Depends. Tolerate a
missing debian/files (first build in a fresh tree) but propagate real
read errors, and hard-error on an unreadable status file like the
source-build path does. installed_build_depends_from_content also
returned a bare newline for zero entries, defeating render_buildinfo's
empty-guard and emitting a malformed 'Installed-Build-Depends:' field
with a blank continuation; it now returns an empty string so the field
is omitted.
2026-09-17 18:44:33 +02:00
vhaudiquet 06e591c665 build: fail binary-only metadata when the previous version is unparseable
The binNMU path swallowed parse errors with a let-chain: a changelog
that could not yield the previous version silently produced a .changes
with plain 'Source: pkg', no Binary-Only-Changes and no redistributed
previous .dsc. Propagate the parse error like the source-build path
does (a single-entry changelog stays tolerated), and reuse the
changelog already read instead of reading the file a second time.
2026-09-17 18:31:31 +02:00
vhaudiquet 12828f8498 deps: resolve :native against DEB_BUILD_ARCH in cross builds
CheckOpts had no build-arch concept: the build-side facts and :native
qualifiers resolved against the host arch, so in a cross build
(-a armhf on amd64) 'Build-Depends: foo:native' looked for an armhf
package where dpkg-checkbuilddeps looks for an amd64 one. CheckOpts
gains build_arch (DEB_BUILD_ARCH), used for :native and the dpkg status
attribution; bracketed arch restrictions keep evaluating against the
host arch.
2026-09-17 18:17:08 +02:00
vhaudiquet eaf1b40369 new: stream orig downloads, reap children on every path
The release-tarball download capped the whole request at 30 s (large
tarballs on slow links always failed and fell through to worse origins)
and buffered the entire body in memory: keep a 10 s connect timeout
only and stream the body to the temp file. The bzip2 -dc child of a
failing repack was neither killed nor waited on (zombie + open pipe);
it is now reaped on both paths. git archive no longer pipes a stderr
nobody drains (a chatty git deadlocked the archive) and any failure
after the destination file was created removes the empty or partial
tarball.
2026-09-17 18:03:24 +02:00
vhaudiquet 27ab4cb9ad put: fix ssh_config negation semantics and file precedence
Host pattern lists were evaluated per-pattern with 'any', so
'Host * !*.launchpad.net' matched ppa.launchpad.net via the wildcard;
a block now applies only if a positive pattern matches and no negated
one does (OpenSSH's rule). The system ssh_config was read first with
first-obtained-wins, inverting OpenSSH's user-over-system precedence;
the user file is read first now. A Match block also no longer leaks
the previous Host block's match state (its options are ignored until
the next Host).
2026-09-17 17:52:17 +02:00
vhaudiquet 3501096107 new: keep flat-tarball entries when repacking the orig tarball
The repack stripped the first path component of every entry, assuming a
single top-level directory: a flat archive ('tar czf up.tar.gz file1
file2') had all its entries dropped and wrote an accepted-but-empty
orig. The layout is now resolved from the leading entries (a lone
top-level directory is held back until the next entry confirms it as
the archive root or proves the archive flat) and flat entries keep
their whole path under the new top-level directory; classic archives
are repacked exactly as before.
2026-09-17 17:45:25 +02:00
vhaudiquet 174a13df39 new: re-probe wizard defaults when packaging a different directory
The detection + probe pass ran against the cwd before the
source-location question, so answering 'another directory' still
offered the cwd's name, version, description, homepage and license
sniff as defaults (only the orig origin followed the chosen tree).
When the answer redirects the wizard to a different directory, the
detection + probe now run again there, feeding every subsequent
probe-derived default; the originally detected directory is not
re-scanned and explicit flags keep winning.
2026-09-17 17:14:25 +02:00
vhaudiquet dd006f7b80 new: validate the command name before generating files
The command/binary name was accepted verbatim and interpolated into
debian/install, debian/rules, debian/tests/smoke, automake variables,
meson.build and [project.scripts]: a value with a space or quote broke
the install lines and shell snippets, 'my.tool' parsed as a nested TOML
table (silently dropping the console script) and produced non-canonical
automake variable names. Both --command and the wizard answer now go
through a shared validator (lowercase identifier: letters, digits,
+ - . _).
2026-09-17 17:03:43 +02:00
vhaudiquet d5b76ec8d8 new: validate wizard defaults like typed answers
ask_text accepted its default on Enter without running the question's
validator, so a probed upstream version like 1.0-2 or v1.0 sailed
through the whole questionnaire and crashed resolve() at the end, and
an invalid git-derived maintainer default (e.g. 'Name <>') was accepted
verbatim. ask_text now takes the validator and applies it to both typed
answers and the offered default — a default that fails validation is
withheld and an invalid answer re-asks — and all question call sites
(incl. the maintainer loop) route through it.
2026-09-17 17:00:22 +02:00
vhaudiquet 70e375a34d new: let an explicit --lang win over build-system detection
The wizard overwrote cli.lang with the detected ecosystem even when the
user passed --lang, and re-asked the language question in the ambiguous
and skeleton cases despite the documented 'flag > detected > default'
merge order. The flag now short-circuits the language step entirely
(detection stays informational); behavior without the flag is
unchanged.
2026-09-17 16:44:35 +02:00
vhaudiquet 8ad50aaf83 put: check the SFTP close status after uploads
ssh2::File's Drop discards the close-handshake error ('too late to
recover'), so a quota or server-side abort surfacing in the final ACKs
was recorded as a successful upload of a truncated file. Close upload
handles explicitly and propagate the error; also applies to the ssh
context driver's write_file and upload_recursive, which had the same
silent-drop issue.
2026-09-17 16:36:36 +02:00
vhaudiquet efb18bfa37 new: fix dead patterns in the generated debian/.gitignore
Patterns containing a slash are anchored relative to the directory
holding the .gitignore, so 'debian/files' inside debian/.gitignore
only ever matched debian/debian/files: every generated pattern was
dead and debhelper artifacts showed up as untracked. Write the
patterns relative to debian/ instead.
2026-09-17 16:33:02 +02:00
vhaudiquet 57db98d776 put: expand ~ in ssh_config IdentityFile paths
IdentityFile values were stored verbatim, so the near-universal
'IdentityFile ~/.ssh/key' spelling never matched an existing file and
the key was silently skipped during authentication. Expand a leading
~ (only that form; ~user and embedded tildes stay verbatim) against
the user's home directory when parsing.
2026-09-17 16:31:04 +02:00
vhaudiquet 775e3d3b8a fmt 2026-09-17 16:27:34 +02:00
vhaudiquet a7cd4244b2 test: hide progress spinners in test runs
Steady-tick spinner threads redraw straight to the real stderr, bypassing
both the harness capture and the per-test log files: 'Scaffolding' lines
from the pkh new tests kept leaking between test results. The scaffold
tests now pass a hidden draw target (the only MultiProgress not created
by the CLI).
2026-09-17 15:32:55 +02:00
vhaudiquet d2bb311f74 net: retry empty index bodies and report by-hash failures in the error
CDNs occasionally answer 200 with a zero-byte body under load; the
checksum verification then reported the empty-string hash as a mismatch,
and the by-hash retry (subject to the same glitch) silently lost its own
failure reason. Treat empty bodies as transient in both fetch paths and
append the by-hash failure to the final VerifyError.

Includes a regression test serving an empty 200 followed by a valid body
on a local socket.
2026-09-17 15:32:55 +02:00
vhaudiquet 4c26122357 test: keep cargo test output quiet with per-test logs and a failure matrix
cargo test used to be unreadable: subprocesses inherited the terminal, so
dpkg-buildpackage, apt and configure output interleaved with the harness
summary, and env_logger lines from parallel tests crossed each other.

New test_support module, compiled into test binaries only (inert stubs
otherwise) and initialized before main via .init_array:

- all log output goes to target/pkh-test-logs/<test>.log, one file per
  test thread, so concurrent tests never interleave
- context-launched commands are captured line by line into the same file
  (driver-level wrapper); test-code spawns use run_logged()
- a panic hook records failures and an atexit callback prints a matrix
  (test name, panic location, message, log path) after the libtest
  summary; tests panicking on purpose can opt out with a guard

Also fixes two test bugs found on the way:

- diff_checkbuilddeps_matrix compared dpkg-checkbuilddeps diagnostics
  against English messages without pinning the locale
- run_source_build in differential tests now captures output like the
  live-UI path does
2026-09-17 15:18:23 +02:00
vhaudiquet 3ed95725e4 net: retry flaky archive fetches and pin index downloads via by-hash
Busy mirrors and CDNs routinely break bulk fetches: pooled keep-alive
connections get closed remotely ('error sending request'), downloads are
cut short (surfacing as bogus checksum mismatches), and index generations
momentarily drift from the Release file fetched moments before.

- shared client: short idle-pool timeout and TCP keepalive, and a
  bounded-retry GET helper now used for index, Release, keyring and
  Launchpad fetches (previously reqwest::get, which has no timeouts)
- downloads: retry the whole download, and check the content length so
  truncation is reported as such instead of a checksum mismatch
- sources index: on a checksum mismatch against the Release file, retry
  pinned to the exact listed generation via Debian's by-hash mechanism;
  body-read errors are retried and reported per component instead of
  aborting the whole lookup
2026-09-17 15:18:14 +02:00
vhaudiquet afedde1f2b new: keep generated builds away from local build outputs and vendored autotools files
CI / build (push) Failing after 2m55s
CI / test (push) Skipped
CI / snap (push) Skipped
2026-09-17 11:33:33 +02:00
vhaudiquet bc3d07abaa new: ignore vendored rust artifacts in the generated gitignore 2026-09-17 10:37:58 +02:00
vhaudiquet bac82f0afe new: skip the git-init question inside existing repositories 2026-09-17 10:37:54 +02:00
vhaudiquet 05e7c55d32 new: detect downloaded tarballs by magic bytes, not extension 2026-09-17 02:15:23 +02:00
vhaudiquet f055b70281 deb: skip quilt patch application for single-debian-patch trees 2026-09-17 02:15:15 +02:00
vhaudiquet 77420e723a new: add upstream-aware orig tarball origins and the orig-vendor component 2026-09-17 01:31:35 +02:00
vhaudiquet 8e06b2074d new: flag rust-toolchain.toml pins in pkh new 2026-09-16 23:59:16 +02:00
vhaudiquet 84824f61c6 new: surface cargo vendor failures and pin the vendoring toolchain 2026-09-16 23:24:43 +02:00
vhaudiquet 9cb3e29a3e new: add Standards-Version to generated control 2026-09-16 22:52:21 +02:00
vhaudiquet c0b7e341fc ui: cleaner transient and completion output for pkh put
CI / build (push) Failing after 2m51s
CI / test (push) Skipped
CI / snap (push) Skipped
2026-09-16 22:43:09 +02:00
vhaudiquet e8d4b98f52 put: verify the changes signature locally with gpgme 2026-09-16 22:24:53 +02:00
vhaudiquet 0c2cf0ac5e put: refuse uploads superseded by published PPA versions 2026-09-16 21:51:33 +02:00
vhaudiquet f27d27ea99 new: add pkh put, a native dput replacement for PPA uploads
Upload built source packages over SFTP with host-key verification
(Launchpad fingerprints pinned in host_keys.yml, ask-to-accept
otherwise), Launchpad account discovery (git config lp.user), and
pre-flight checks the upload queue itself never does: changes file
discovery/validation, PPA existence via the Launchpad API, target
series validity, and debian/control Section validity (sections
bundled in distro_info.yml). Upload log prevents duplicate uploads
unless --force.
2026-09-16 21:38:53 +02:00
vhaudiquet 9228ff448b new: give wizard select labels their own separator
CI / build (push) Failing after 2m47s
CI / test (push) Skipped
CI / snap (push) Skipped
2026-09-16 14:01:23 +02:00
vhaudiquet 9b98f5c7c3 new: add interactive wizard and remaining ecosystem templates 2026-09-16 13:49:29 +02:00
vhaudiquet d044f757e9 new: scaffold new Debian source packages (non-interactive core) 2026-09-16 12:14:09 +02:00
vhaudiquet 9c3394750d distro_info: accept case-insensitive dist in effective_series, fix UNRELEASED fallbacks 2026-09-16 12:12:15 +02:00
vhaudiquet 60976d3feb ui: generalize interactive prompts into ui::prompt 2026-09-16 11:15:31 +02:00
vhaudiquet ae420989f9 distro_info: add UNRELEASED series handling, use it in deb and chlog 2026-09-16 11:15:09 +02:00
vhaudiquet 4af8dbddb0 fmt: fix clippy warnings on all targets
CI / build (push) Successful in 2m48s
CI / test (push) Skipped
CI / snap (push) Failing after 11s
Drop an unused test fixture constant, move download_trust_ppa_key above
the test module that precedes it, and simplify two test borrows.
2026-09-16 09:15:52 +02:00
vhaudiquet 3a454b0811 build: record the actual build environment in .buildinfo
The binary build exported DEB_BUILD_OPTIONS='parallel=<context nproc>
nocheck' (or the -j override) but the generated .buildinfo recomputed
the environment from host state: host core count, no nocheck, and
vendor profiles that ignored DEB_BUILD_PROFILES (a cross build recorded
no 'cross' profile). generate_binary_metadata now records the exact env
map that was exported to the build steps, and the recorded profiles
come from the exported DEB_BUILD_PROFILES when set.

Also unifies vendor parsing on one helper (the context-side copy lacked
the Origin: fallback of the source-build path).
2026-09-16 04:22:43 +02:00
vhaudiquet 50ae12cafe prune: match real log names, order retention by time, spare fresh locks
Log retention only matched 'deb-*' logs, so source-build ('build-*')
and placeholder ('pkh-*') logs accumulated forever, and the 'keep the
newest' sort was lexicographic on names that sort by package/version
first, so arbitrary logs were kept. All three log shapes are matched
now and retention orders by the timestamp embedded in the name (mtime
fallback). Stale-lockfile pruning no longer deletes lockfiles younger
than 24h: a fresh <tarball>.lock is the mutual-exclusion signal of a
concurrent download and deleting it could corrupt the shared tarball
cache.
2026-09-16 04:04:50 +02:00
vhaudiquet 38562abe2c debian: fix deb822 writer/parser asymmetries
write_paragraph emitted a bare-space continuation line for empty lines
inside a value, which parse_paragraphs treated as a paragraph separator
and silently dropped the rest of the value; blank lines are now encoded
as ' .' like dpkg does and decoded back on read. Tab-indented
continuation lines now strip exactly one tab instead of keeping it.

Clearsigned .dsc content no longer leaks armor metadata into parsed
fields: the Hash:/Comment: header and the signature trailer are
stripped before parse_paragraphs at both .dsc parse sites.
2026-09-16 03:42:30 +02:00
vhaudiquet dc6a019a13 build: parse .dsc checksum fields through one shared, validating parser
The source-build pipeline and the binNMU metadata path had drifted into
two inline parsers with different acceptance rules: binary.rs filled
names from any line with a third column but partials only from
exactly-three-column lines, so a 4+ column Checksums line made
&partials[name] panic by map index. Both paths now share one parser
that accepts the modern 3-column and the legacy 5-column Files layout,
rejects anything else with an error naming the field and line, and all
remaining lookups go through .get() with a clear error instead of
indexing. Legacy 5-column Files md5s were previously attributed to the
section token instead of the file name.
2026-09-16 03:20:01 +02:00
vhaudiquet f72b35acfa Handle malformed remote and edge-case data instead of panicking
- distro_info: malformed CSV rows are skipped with a warning, dates
  that fail to parse become None, and all plain HTTP requests go
  through a shared reqwest client with connect/total timeouts
- package_info: the Sources stanza iterator is iterative (a crafted
  index with many blank stanzas overflowed the stack), stanzas missing
  a Version are skipped, and failed series/pocket probes are summarized
  in the final 'not found' error instead of being silently dropped
- pull: no double unwrap on the remote-derived artifact filename, an
  empty series list is an error, and streaming downloads get a
  per-request timeout
- deb/cross: dpkg-architecture output parsing skips unexpected lines
  and its exit status is checked, as is dpkg --add-architecture
- changelog: version increments parse as u64 with checked arithmetic
  (1.0-20250123123456 used to panic on the u32 parse)
2026-09-16 02:44:45 +02:00
vhaudiquet 6a5c5a7106 context: fix deadlock and panics when resolving parented contexts
set_current held the config RwLock for writing across make_context,
which for a context with a parent re-entered the same lock through
Context::new's global-manager lookup, deadlocking 'pkh context use'.
Context building is now lock-free by construction: make_context
resolves parent chains against a snapshot map (also rejecting parent
cycles), and neither set_current nor remove_context holds a guard
while building a Context.

A corrupt contexts.json no longer aborts every command at manager
init: load falls back to the default local-only config, backs the
corrupt file up to contexts.json.bak so a later save cannot silently
destroy it, and a dangling current/parent context falls back to local
with an error log instead of panicking.
2026-09-16 02:27:11 +02:00
vhaudiquet 592e98c1e9 deb: pass the build context explicitly instead of swapping the global
build_binary_package installed its ephemeral chroot context into the
process-global manager and read it back with context::current(),
ignoring its ctx parameter: two concurrent builds would re-point each
other's global and each drop would clean up whichever chroot was
current at the time. The guard now keeps the Arc of the context it
created (parented directly on the base context, not on a config-name
lookup), exposes it via context(), and Drop cleans up exactly that
context and restores the exact handle that was current at creation,
so overlapping builds no longer cross-destroy each other.
2026-09-16 02:05:40 +02:00
vhaudiquet 512a1cb778 deb: unmount and remove the ephemeral chroot on Ctrl-C
The SIGINT handler libc::_exit(130)s, skipping EphemeralContextGuard's
drop and leaking the freshly bootstrapped chroot with its bind-mounted
/proc and overlay mounts. Resources now register a self-contained
cleanup hook in a process-global registry that the handler drains right
before exiting: the hook unmounts everything under the chroot path
(children first, lazy fallback) and removes the tree, using only stored
paths and direct umount/rm subprocesses so it cannot deadlock on a lock
the interrupted thread may hold; sudo -n keeps it from ever hanging on
a password prompt. Drop deregisters the hook first, so the normal
cleanup path is unchanged.

Also fixes the hex grouping of the CRC-24 polynomial in apt::release.
2026-09-16 01:55:35 +02:00
vhaudiquet 93176aa479 deps: give legacy < and > relations their documented dpkg semantics
Debian Policy 7 defines the deprecated single-character spellings as
'earlier/later or equal' (i.e. <= and >=), and dpkg still accepts them
that way; the parser mapped them to the strict << and >> instead, so
'foo (< 1.0)' was wrongly reported unmet against installed 1.0.
2026-09-16 01:30:25 +02:00
vhaudiquet 60ee99adc8 pull: search pockets in release order by default
Without an explicit --pocket, find_package stops at the first pocket
containing the package, but the search order listed '-proposed' first
and never included '-security': unreleased proposed packages won by
default and security-only updates were unreachable. Search the main
archive first, then updates, security, and proposed last.
2026-09-16 01:27:25 +02:00
vhaudiquet f6fed7328b pull: reject malformed --ppa values instead of silently using the archive
A --ppa value that was not exactly 'user/name' (full URL, extra
segment, empty halves) made base_url None and pulled the package from
the main archive without any warning. Error out naming the expected
format instead, and document the format in --help.
2026-09-16 01:24:02 +02:00
vhaudiquet 213668fa82 pull: authenticate archive indexes against signed Release files
The Sources index was downloaded with no authentication: per-artifact
checksums were verified, but against hashes taken from an index a MITM
could substitute along with the artifacts. Fetch each suite's InRelease
(or Release + Release.gpg), verify the signature with gpgv against the
archive keyring (or the PPA signing key) the same way apt does, and
checksum-check every Sources index against it before parsing.

Distro archives and PPAs verify strictly: an invalid or unverifiable
signature, or a missing gpgv binary, is a hard error. Flat repositories
keep working without a Release file or without a verifiable one (warned
as unauthenticated), but tampering evidence is a hard error there too.

Also switches all archive, PPA and keyring base URLs to https, and
reads suite components from the verified Release instead of fetching
them separately over an unauthenticated channel.
2026-09-16 01:22:22 +02:00
vhaudiquet f508f20846 context: quote program, args, cwd and env in ssh, schroot and unshare drivers
All three non-local drivers assembled remote/chroot command strings by
raw concatenation: ssh pushed args verbatim (TODO: escape), schroot
interpolated env values raw so DEB_BUILD_OPTIONS='parallel=4 nocheck'
made sh treat 'nocheck' as the command, and unshare wrapped args in
unescaped double quotes letting quotes break out and $/backticks
expand. Add a shared POSIX shell_quote helper and use it for every
component interpolated into a shell string, including ssh copy_path
(which used Rust's {:?}, not shell quoting) and schroot write_file
(which also switches echo -ne to printf %s so backslash sequences in
content are no longer interpreted).
2026-09-16 00:36:02 +02:00
vhaudiquet 685538e637 apt: round-trip sources in place instead of consolidating them into sources.list
Saving the modified source entries with save_legacy rewrote every entry
into /etc/apt/sources.list in legacy format, destroying deb-src entries
and signed-by/trusted options, duplicating every distro entry that came
from a deb822 file (which stayed in place), and hardcoding the Ubuntu
keyring on cross builds. Entries now remember the file and format they
were loaded from and are written back there; new entries (PPAs, ports)
go to a pkh-owned /etc/apt/sources.list.d/pkh-added.list, and a one-time
<path>.pkh-backup copy is made before overwriting an existing file.

Also fixes: 'Types: deb deb-src' stanzas are split instead of being
treated as binary-only, commented-out legacy entries are kept disabled
instead of deleted, debian.sources is actually read on Debian (the old
else-if never fired), and the double blank lines save_legacy emitted.
2026-09-16 00:10:45 +02:00
vhaudiquet ea70ddc10d apt: keep the keyring cache private to the invoking user
The shared world-writable /tmp/pkh-keyrings directory, combined with the
skip-if-exists logic, let any local user pre-plant keyrings that pkh
then trusts into the chroot's trusted.gpg.d. Use a per-uid 0700
directory instead, refuse to reuse a pre-existing directory that is not
owned by the current user or is group/other-writable, and drop the now
unnecessary world-accessibility chmods (mmdebstrap in unshare mode runs
with the same real uid).
2026-09-15 23:47:21 +02:00
vhaudiquet 48f6e6ce4e pull: confine tar extraction to the destination directory
Entry::unpack performs no path sanitization, so a malicious or malformed
tarball (PPA, flat repository) could write files outside the package
directory via '..' components or absolute entry paths. Refuse such
entries with an error naming the offending path.
2026-09-15 23:30:28 +02:00
vhaudiquet b34e86dcfe deb: scope the arch-indep build-dep pass to the host arch in cross builds
CI / build (push) Successful in 2m49s
CI / test (push) Skipped
CI / snap (push) Failing after 11s
Without --host-architecture, the second build-dep pass re-resolves the
whole Build-Depends field for the native architecture: apt swaps
host-arch -dev packages for native ones (e.g. libcurl4-gnutls-dev,
whose arch-differing curl-config makes dpkg refuse the co-install) and
breaks the cross build environment.

Per dpkg-checkbuilddeps, both Build-Depends and Build-Depends-Indep
resolve for the host architecture in cross mode, so pass
--host-architecture to the second pass as well. Skip the pass entirely
when the source declares no Build-Depends-Indep.

Add an end-to-end regression test building a package that declares
libdb-dev in both fields and links a host-arch binary against it: the
test only passes if the arch-indep pass did not swap the arm64 -dev
packages for native ones.
2026-09-15 18:05:57 +02:00
vhaudiquet 5500f98586 pull: add --repository to pull from external flat repositories
Add a --repository flag taking the full suite URL of an external flat
repository (e.g. https://pkg.noctalia.dev/deb/resolute/), i.e. one with
no dists/ hierarchy, like apt's exact-path suites ('Suites: resolute/').

The suite name is read from the root Release file (Codename/Suite), the
sources index is fetched from the repository root as Sources.xz/gz/plain,
and package files are resolved against the URL root, ignoring the stanza
Directory field like apt does. As with PPAs, the stanza Vcs-Git is never
used for external repositories, so the source always comes from the
repository itself.

Also make the sources index parser detect compression by magic bytes
(gz/xz/plain) instead of assuming gzip, and fix extraction of archives
with './'-prefixed entries, which previously aborted and are now
extracted in place instead of being relocated.
2026-09-15 10:57:06 +02:00
vhaudiquet 3b99ece39a fmt
CI / build (push) Successful in 2m46s
CI / test (push) Skipped
CI / snap (push) Failing after 28s
2026-09-10 15:31:17 +02:00
vhaudiquet c7af3bc9b1 deb: retrieve only the artifacts produced by the build, not globbed files
CI / build (push) Failing after 53s
CI / test (push) Skipped
CI / snap (push) Skipped
build_binary_package_impl copied the whole parent directory into the
build root (ensure_available) and then retrieved every *.deb / *.changes
/ *.buildinfo it found there. That surfaced stale files already sitting
next to the package tree in the "Built in Ns:" summary.

Now local::build returns the exact set of artifacts produced by this
build — the binary packages registered in debian/files plus the
generated .buildinfo/.changes from generate_binary_metadata — and
build_binary_package_impl retrieves that list instead of globbing the
build root. Only files genuinely produced by the current build are
printed.
2026-09-10 15:30:06 +02:00
vhaudiquet 182a06ffbe deb: add -j/--jobs to control parallel build jobs
By default the number of parallel jobs is detected with nproc inside
the build context. Add a -j/--jobs option so an explicit count can be
honored instead, threading it through build_binary_package and
local::build into DEB_BUILD_OPTIONS=parallel=N.
2026-09-10 15:27:50 +02:00
128 changed files with 37834 additions and 2478 deletions
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 799 KiB

+43 -2
View File
@@ -3,6 +3,7 @@ name: CI
on:
push:
branches: [ "main", "ci-test" ]
tags: [ "v*" ]
pull_request:
branches: [ "main" ]
@@ -102,7 +103,7 @@ jobs:
- name: Install build prerequisites
run: |
apt-get update -q
apt-get install -y -q --no-install-recommends git curl
apt-get install -y -q --no-install-recommends git curl nodejs
- uses: actions/checkout@v6
- name: Build snap
run: |
@@ -114,8 +115,48 @@ jobs:
export PATH="/usr/libexec/snapcraft:$HOME/.cargo/bin:$PATH"
snapcraft pack --destructive-mode
- name: Upload snap artifact
uses: actions/upload-artifact@v4
# v4 refuses to run outside github.com (GHESNotSupportedError); Gitea
# implements the artifact API used by v3, so v3 is the supported choice.
uses: actions/upload-artifact@v3
with:
name: snap
path: ./*.snap
if-no-files-found: error
publish:
# Publishes the crate to crates.io on a v* tag. Trusted publishing
# (OIDC) is GitHub-Actions-only, so authentication goes through a
# crates.io API token stored as the CARGO_REGISTRY_TOKEN secret,
# scoped to the pkh crate.
if: startsWith(github.ref, 'refs/tags/v')
needs: build
runs-on: ubuntu-latest
container:
image: ubuntu:26.04
options: --privileged --cap-add SYS_ADMIN --security-opt apparmor:unconfined
steps:
- name: Set up container image
run: |
apt-get update
apt-get install -y nodejs sudo curl wget ca-certificates build-essential
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
- name: Install build dependencies
run: |
sudo apt-get update
sudo apt-get install -y pkg-config libssl-dev libgpg-error-dev libgpgme-dev
- name: Check the tag matches the crate version
# cargo publish ships whatever version Cargo.toml declares,
# regardless of the tag: a mismatch must fail loudly instead of
# publishing the wrong version under the release tag.
run: |
crate_version="$(awk -F'"' '/^version =/{print $2; exit}' Cargo.toml)"
tag_version="${GITHUB_REF_NAME#v}"
if [ "$crate_version" != "$tag_version" ]; then
echo "tag $GITHUB_REF_NAME does not match crate version $crate_version" >&2
exit 1
fi
- name: Publish
run: cargo publish
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
+7 -1
View File
@@ -1,2 +1,8 @@
*.lock
target
# Local snapcraft builds
.craft
parts
prim
stage
*.snap
+108
View File
@@ -0,0 +1,108 @@
# AGENTS.md
Conventions for working in this tree. They apply to every commit; the
whole history follows them.
## Before every commit
Run, in order, and make sure they are clean before committing:
```sh
cargo fmt --all
cargo clippy --all-targets --all-features # zero warnings
```
CI builds and lints with `RUSTFLAGS: -Dwarnings`, so a `cargo build`
warning fails the gate too. `cargo fmt` may amend files you did not
touch — include those changes in the commit (or in a separate `fmt:`
commit) rather than leaving the tree dirty.
The test suite is heavy (chroots, ssh, network): always run the
`#[cfg(test)]` modules of what you touch, and the full suite when
changing shared plumbing (`report`, `logfmt`, `test_support`,
`debian/`). Tests marked `#[ignore]` shell out or hit the network and
are for deliberate ad-hoc runs (`cargo test -- --ignored`), not for the
pre-commit pass. Building needs the gpgme/openssl system packages
(`pkg-config libssl-dev libgpg-error-dev libgpgme-dev`).
## Commit messages
There are no conventional-commit types; the format is a component scope
and a summary:
```
<scope>: <short summary>
```
Rules:
- The scope is the component touched — the module under `src/` (file or
directory), named after the user-facing subcommand when that differs:
- `pull` — source package download (`src/pull.rs`)
- `chlog` — changelog entry generation (`src/changelog.rs`)
- `build` — source package builds, .dsc (`src/build/`)
- `deb` — binary package builds, .deb (`src/deb/`)
- `put` — PPA/archive upload (`src/put/`)
- `new` — package scaffolding (`src/new/`)
- `lint` — tree linting (`src/lint/`)
- `prune`, `package_info` — remaining subcommand modules
- `context` — build contexts: local, ssh, chroot/schroot, unshare
(`src/context/`)
- `interrupt` — Ctrl+C interception and interrupt-time cleanup
(`src/interrupt.rs`)
- `debian` — Debian format primitives: control, versions, checksums,
arch (`src/debian/`)
- `apt`, `launchpad`, `distro_info`, `quirks` — archive/distro
integration
- `report` — BuildView/Prompter ports and the views implementing them
- `ui`, `logfmt` — terminal rendering and output classification
- `data` — the `data/*.yml` embed convention itself; content changes
to a data file belong to the commit of the module consuming it
- `cli` — the binary, argument wiring (`src/main.rs`)
- `test` — test-only changes (shared plumbing: `src/test_support.rs`)
- `deps` — dependency additions/bumps (manifests, lockfile)
- `fmt`, `clippy` — rustfmt/clippy fixups
- `ci`, `snap`, `docs` — workflows, snap packaging, README
- Use the submodule path when the change is confined to one
(`apt/keyring`, `debian/version`).
- A commit touching several components should be split into one commit
per component when practical; otherwise comma-join the scopes without
spaces (`pull,deb`).
- Summary: imperative mood, lowercase first letter (proper nouns keep
theirs: Ubuntu, SRU, lintian), no trailing period, max ~72 characters.
- Body (expected for anything nontrivial): separated by a blank line,
wrapped at 72 columns; explain why, and the design when the approach
was a choice among alternatives. Reference issues as `#123`.
- Reverts use git's default `Revert "<original subject>"`.
### Examples
```
chlog: fall back to the changelog history when no version tag exists
ui: ellipsize fake-terminal pane lines wider than the terminal
lint: add pkh lint, wrapping lintian for parity plus pkh-native checks
chlog: number Ubuntu backports with the per-release SRU scheme
deb: resolve cross pkg-config against the target multiarch
pull,deb: add top-level --pocket option
debian/version: dpkg-compatible version comparison
deps: bump git2 to 0.21
fmt: apply rustfmt
docs: refresh the README roadmap for 1.0
```
## Code
- The crate denies missing docs (`#![deny(missing_docs)]` in
`src/lib.rs`): every public item carries a doc comment, and the module
list there is the layout map — keep it in sync when adding a module.
- Subcommand business logic lives in the library and reports through the
`report` ports (`BuildView`, `Prompter`) instead of printing;
`src/main.rs` is argument wiring only. Subprocess output
classification is pure logic in `logfmt`, testable without a pty.
- Static reference data (series tables, keyserver URLs, licenses,
forges, templates) lives in `data/*.yml`, embedded with the
`embed_data!` macro — not in hardcoded tables.
- Comments state constraints the code cannot show; no narration.
- Anything user-facing (subcommands, flags, option defaults) is
reflected in `README.md` — including its roadmap checklists — before
commit.
Generated
+3068
View File
File diff suppressed because it is too large Load Diff
+7
View File
@@ -3,6 +3,10 @@ name = "pkh"
version = "0.1.0"
edition = "2024"
authors = ["vhaudiquet"]
description = "pkh is a packaging helper for Debian/Ubuntu packages"
license = "MIT OR GPL-2.0-only"
repository = "https://git.vhaudiquet.fr/vhaudiquet/pkh"
readme = "README.md"
[dependencies]
clap = { version = "4.5.51", features = ["cargo"] }
@@ -34,6 +38,9 @@ ssh2 = "0.9.5"
gpgme = "0.11"
serde_yaml = "0.9"
lazy_static = "1.4.0"
unicode-width = "0.2"
parking_lot = "0.12"
suppaftp = "12"
[dev-dependencies]
test-log = "0.2.19"
+338
View File
@@ -0,0 +1,338 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
<https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Lesser General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Moe Ghoul>, 1 April 1989
Moe Ghoul, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License.
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025-2026 Valentin Haudiquet
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+44 -55
View File
@@ -2,6 +2,30 @@
`pkh` is a packaging helper for Debian/Ubuntu packages.
![](.github/pkh.gif)
## Installation
From crates.io:
```
cargo install pkh
```
Or build from source (the same system packages are needed either way):
```
sudo apt install pkg-config libssl-dev libgpg-error-dev libgpgme-dev
git clone https://git.vhaudiquet.fr/vhaudiquet/pkh.git
cd pkh
cargo install --path .
```
At runtime pkh shells out to the Debian packaging toolchain (git,
dpkg-dev, quilt, mmdebstrap, lintian, pristine-tar, ...): install the
ones your workflows use, or build the classic snap from
`snap/snapcraft.yaml` (`snapcraft pack`), which carries them.
## Usage and features
### Basic concepts
@@ -25,11 +49,19 @@ Options:
Commands and workflows include:
```
Commands:
new Scaffold a new Debian source package (buildable right away)
pull Pull a source package from the archive or git
chlog Auto-generate changelog entry, editing it, committing it afterwards
build Build the source package (into a .dsc)
put Upload the built source package to a PPA
deb Build the source package into binary package (.deb)
lint Lint the package (lintian wrapper + pkh-native checks)
prune Prune residual pkh build artifacts and caches
help Print this message or the help of the given subcommand(s)
Options:
-h, --help Print help
-V, --version Print version
```
### Examples
@@ -65,68 +97,25 @@ That is a lot of different tools and operations. With pkh, the same workflow:
pkh pull hello # needs -d ubuntu if you are not running Ubuntu
# Apply the patch to the package
...
pkh commit -m "Applied patch xxx"
git add debian/patches/xxx.patch
git commit -m "Applied patch xxx"
pkh chlog
git add debian/changelog
git commit -m "d/changelog"
# Test that the package builds
pkh build
pkh deb
# Upload the package to a ppa
pkh put --ppa user/hello_xxx
# Push previously commited changes
# Push the commits to your fork
git push xxx user-fork
```
## Roadmap: features needed for 1.0
Basically, wrapping the basic debian workflows.
Missing features:
- [ ] `pkh pull`
- [x] Obtain package sources from git
- [x] Obtain package sources from the archive (fallback)
- [x] Obtain package source from PPA (--ppa)
- [ ] Obtain a specific version of the package
- [x] Fetch the correct git branch for series on Ubuntu
- [ ] Try to fetch the correct git branch for series on Debian, or fallback to the archive
- [ ] `pkh chlog`
- [x] Auto-generate changelog entry
- [ ] Extra flags: backport, non-maintainer upload, no change rebuild, ...
- [ ] Commit changelog entry
- [ ] `pkh build`
- [x] Build the source package
- [ ] `pkh deb`
- [x] Build the binary package
- [x] Build for a specific architecture
- [ ] Three build modes:
- [ ] Build locally (discouraged)
- [x] Build using unshare chroot, with binary emulation (default)
- [x] Cross-compilation
- [ ] Async build
- [ ] `pkh status`
- [ ] Show build status
- [ ] `pkh put`
- [ ] Upload the source package to a PPA
- [ ] Upload the source package to the archive
- [ ] `pkh commit`
- [ ] Commit the changes to git
- [ ] `pkh lint`
- [ ] Lint the package
- [ ] `pkh test`
- [ ] Run autopkgtest
- [ ] Provide options: local (discouraged), chroot, VM?, ppa
- [ ] Async test
## Nice-to-have features
- [ ] 'pkh pull'
- [ ] Cache the Sources.gz files, to improve speed
- [ ] Work in an already downloaded package, to git pull and re-fetch orig tar gz
- [ ] 'pkh context'
- [x] Select, add, remove, list contexts
- [x] Context-scoped command execution
- [ ] Context-scoped deb and test commands
- [ ] Per-architecture contexts
- [ ] Per-series contexts
- [x] ssh contexts
- [ ] docker, lxc contexts?
- [ ] context push, context pop: context stack
## Future improvement ideas
- pull: try to fetch the correct git branch for series on Debian
- deb: asynchronous build, detachable and monitorable
- put: allow uploads to Debian or Ubuntu archives
- test: add 'pkh test' to run autopkgtests
- pull: cache Sources.gz files to improve speed
- pull: 'pkh pull' in a package tree should git pull and re-fetch orig tgz
+210
View File
@@ -0,0 +1,210 @@
## Static data needed for pkh operations
## Instead of hardcoding the data in code, data files allow to quickly
## update and maintain such data in one unique place
## The goal is to have the minimal possible set of data necessary
## to grab the actual data. For example we don't want to store every Ubuntu
## or Debian series, but rather pointers to where that data lives: each dist
## entry below carries its series sources (the local distro-info CSV, with
## the network URL as fallback).
##
## Per-dist keys beyond the series pointers:
## mirrors: the archive mirrors, each a URL serving a set of
## architectures: `primary` (the main archive, whose url
## doubles as the dist's base URL) and, where they exist,
## the others (`ports`). `security_url` is the sibling
## host serving the -security pocket for the same arches
## (ports mirrors serve their own security); `archs` is
## an explicit list, or the `all` sentinel when one
## mirror serves every architecture (Debian's case — an
## exhaustive list would rot each time an arch is added).
## Host matching treats a URI as official when its host
## equals a mirror host or is a subdomain of it, so the
## country mirrors (fr.archive.ubuntu.com) count too.
## components: the archive components (main, universe, contrib, ...)
## a cross-build environment enables on official sources.
## Live archive operations keep resolving components from
## Release files; this is the offline default.
## cross_pockets: the pockets a cross-build environment enables for a
## series (`<series>-updates`, ...). Deliberately not the
## `pockets` key: that one is the *search order* of pull,
## where backports must not fold in.
## suite_aliases: the changelog suite names that alias a series
## codename: Debian packages conventionally target
## 'unstable' where the series data carries 'sid'.
## Mapped suite name -> series codename; the two
## names identify the same series, and the selector
## offers the aliased entry as '<suite> (<series>)'.
## build_profiles: the vendor's default DEB_BUILD_PROFILES (Ubuntu
## activates derivative.ubuntu noudeb, Debian none),
## mirroring what Dpkg::BuildProfiles resolves when the
## variable is unset.
dist:
debian:
mirrors:
primary:
url: https://deb.debian.org/debian
# One mirror serves every architecture.
archs: all
components: [main, contrib, non-free, non-free-firmware]
cross_pockets: [updates, backports, security]
build_profiles: []
archive_keyring: https://ftp-master.debian.org/keys/archive-key-{series_num}.asc
pockets:
- updates
- security
- proposed-updates
# Debian changelogs conventionally target 'unstable'; the series data
# knows the same series as 'sid'.
suite_aliases:
unstable: sid
sections:
# Valid Section values for debian/control: the Debian policy section
# list unioned with the sections observed in the live Ubuntu archive.
# Archives reject uploads carrying an unknown section; only the part
# before a '/' (the subsection) is validated.
- admin
- cli-mono
- comm
- database
- debian-installer
- debug
- devel
- doc
- editors
- education
- electronics
- embedded
- fonts
- games
- gnome
- gnu-r
- golang
- graphics
- hamradio
- haskell
- httpd
- interpreters
- introspection
- java
- javascript
- kde
- kernel
- libdevel
- libs
- lisp
- localization
- mail
- math
- metapackages
- misc
- net
- news
- ocaml
- oldlibs
- otherosfs
- perl
- php
- python
- ruby
- rust
- science
- shells
- sound
- tasks
- tex
- text
- translations
- utils
- vcs
- video
- web
- x11
- xfce
- zope
series:
local: /usr/share/distro-info/debian.csv
network: https://salsa.debian.org/debian/distro-info-data/-/raw/main/debian.csv
ubuntu:
mirrors:
primary:
url: https://archive.ubuntu.com/ubuntu
# Sibling host serving the -security pocket for the same arches.
security_url: http://security.ubuntu.com/ubuntu
archs: [amd64, i386]
ports:
# Everything else lives on the ports archive, which also serves
# its own -security pocket (no security_url needed).
url: http://ports.ubuntu.com/ubuntu-ports
archs: [armhf, arm64, ppc64el, riscv64, s390x]
components: [main, restricted, universe, multiverse]
cross_pockets: [updates, backports, security]
build_profiles: [derivative.ubuntu, noudeb]
archive_keyring: https://archive.ubuntu.com/ubuntu/project/ubuntu-archive-keyring.gpg
pockets:
- updates
- security
- proposed
sections:
# Same list as debian (see the comment there)
- admin
- cli-mono
- comm
- database
- debian-installer
- debug
- devel
- doc
- editors
- education
- electronics
- embedded
- fonts
- games
- gnome
- gnu-r
- golang
- graphics
- hamradio
- haskell
- httpd
- interpreters
- introspection
- java
- javascript
- kde
- kernel
- libdevel
- libs
- lisp
- localization
- mail
- math
- metapackages
- misc
- net
- news
- ocaml
- oldlibs
- otherosfs
- perl
- php
- python
- ruby
- rust
- science
- shells
- sound
- tasks
- tex
- text
- translations
- utils
- vcs
- video
- web
- x11
- xfce
- zope
series:
local: /usr/share/distro-info/ubuntu.csv
network: https://salsa.debian.org/debian/distro-info-data/-/raw/main/ubuntu.csv
+31
View File
@@ -0,0 +1,31 @@
## Forge hosts recognized by `pkh new` origin detection, with the
## release-tarball URL templates of each: `Forge::parse`
## (src/new/origin.rs) matches a git remote's host against the map keys,
## and the tarball download substitutes {owner}, {repo} and {tag} into the
## templates. Like host_keys.yml, this file exists so that static
## endpoints are data: adding a forge is a YAML entry, not a code change
## (self-hosted instances are deliberately absent — the download URL
## shapes differ per instance).
##
## tarball_templates are tried sequentially in file order, best candidate
## first (GitHub prefers the codeload direct link: no redirect).
## `kind` documents the forge family the URL shapes belong to; the
## templates fully describe the URLs, so nothing branches on it (yet) —
## but it must be one of the known kinds, enforced at load time.
##
## Where the values come from: each forge's release-archive download URL
## shapes, verified against the live forges —
## github: codeload.github.com/<owner>/<repo>/tar.gz/refs/tags/<tag>
## and github.com/<owner>/<repo>/archive/refs/tags/<tag>.tar.gz
## gitlab: gitlab.com/<owner>/<repo>/-/archive/<tag>/<repo>-<tag>.tar.gz
forges:
github.com:
kind: github
tarball_templates:
- https://codeload.github.com/{owner}/{repo}/tar.gz/refs/tags/{tag}
- https://github.com/{owner}/{repo}/archive/refs/tags/{tag}.tar.gz
gitlab.com:
kind: gitlab
tarball_templates:
- https://gitlab.com/{owner}/{repo}/-/archive/{tag}/{repo}-{tag}.tar.gz
+19
View File
@@ -0,0 +1,19 @@
## SSH host key fingerprints of known upload targets (`pkh put`).
## Like distro_info.yml, this file exists so that trust anchors are data,
## quickly updatable in one place, instead of hardcoded in the source.
##
## A server presenting a key whose fingerprint is listed for its host is
## verified without prompting. Fingerprints are the `SHA256:<base64>` values
## as displayed by ssh-keygen / pkh; an optional key type prefix (e.g.
## `ssh-rsa`) is tolerated as the first word of an entry.
##
## Source of the Launchpad fingerprints (published "as a stopgap measure
## until we have signed DNS records"):
## https://ubuntu.com/docs/launchpad/user/reference/ssh-fingerprints/
## (formerly https://help.launchpad.net/SSHFingerprints)
fingerprints:
ppa.launchpad.net:
- ssh-rsa SHA256:MGq+4hxD7RduVTcfwlwwboZnsgJC6SL/NltM8ye+gNg
upload.ubuntu.com:
- ssh-rsa SHA256:FN8sNU/MMmyvw/xtY5sAzkLGmkVQt2QpGZcwsHoBzjc
+17
View File
@@ -0,0 +1,17 @@
## Keyserver lookup endpoint used to fetch PPA signing keys.
## Like host_keys.yml, this file exists so that a static endpoint is data,
## updatable in one reviewable place, instead of hardcoded in the source —
## the URL was previously duplicated in the apt keyring and release
## modules. Sparse on purpose: it grows if keyserver pools or alternates
## ever need to be tried.
##
## The template carries its variable part as a {fingerprint} placeholder,
## substituted by the accessor of src/apt/keyring.rs with plain string
## replacement.
##
## Where the value comes from: keyserver.ubuntu.com, Ubuntu's OpenPKS
## (formerly SKS) keyserver; op=get with search=0x<fingerprint> is the
## documented machine interface fetching one key by fingerprint
## (https://keyserver.ubuntu.com).
lookup_template: "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x{fingerprint}"
+33
View File
@@ -0,0 +1,33 @@
## Launchpad service endpoints: the REST API, the PPA SFTP upload queue,
## the PPA package-content host and the Ubuntu source-package git web UI.
## Like host_keys.yml, this file exists so that static endpoints are data,
## updatable in one reviewable place, instead of hardcoded in the source —
## the API and content URLs were previously triplicated across modules.
##
## Templates carry their variable parts as {name} placeholders ({owner},
## {ppa}, {package}), substituted by the accessors of src/launchpad.rs
## with plain string replacement.
##
## Where the values come from:
## api_base: the Launchpad REST API root (https://launchpad.net/docs/api/)
## ssh_*: the PPA upload queue, as expanded by dput-ng's
## ppa:user/ppa profile (ppa.launchpad.net:22, incoming
## ~<user>/<ppa>)
## ftp_*: the same upload queue over anonymous FTP, dput-ng's
## plain ppa: profile: the transport pkh degrades to
## when the SSH connection itself never comes up
## content_host_template: ppa.launchpadcontent.net serves PPA apt
## repositories since the 2022 move off ppa.launchpad.net
## git_web_template: Launchpad's CGit mirrors of Ubuntu source packages
## (git.launchpad.net/ubuntu/+source/<package>)
api_base: https://api.launchpad.net/1.0
ssh_host: ppa.launchpad.net
ssh_port: 22
## The anonymous FTP upload queue dput-ng's plain ppa: profile uses:
## pkh degrades to it when the SSH connection itself never comes up.
ftp_host: ppa.launchpad.net
ftp_port: 21
incoming_template: "~{owner}/{ppa}"
content_host_template: https://ppa.launchpadcontent.net/{owner}/{ppa}/ubuntu
git_web_template: https://git.launchpad.net/ubuntu/+source/{package}
+139
View File
@@ -0,0 +1,139 @@
## License knowledge of `pkh new`, in one place: the wizard menu labels
## (src/new/questions.rs), the spellings accepted by License::parse and the
## SPDX URL template (src/new/options.rs), and the license-file sniffing
## inputs (src/new/detect.rs) all read this table, so the three lists —
## previously kept in sync by comments only — cannot drift apart anymore.
## Adding or changing a curated license is one entry below.
##
## Keys:
## id: SPDX identifier: written to debian/copyright, returned
## by the license sniff and substituted into
## license_url_template (minus a trailing '+' of the
## "or later" spellings)
## menu: label offered by the wizard license question (the
## free-text "Other (enter a SPDX identifier)" entry
## stays in Rust — it is UX, not data)
## spellings: inputs accepted by License::parse, matched
## case-insensitively; each entry must accept its own id
## (a consistency test in options.rs locks ids, spellings
## and the License enum together)
## detect_markers: marker sets driving the LICENSE/COPYING text sniff of
## detect.rs. A set matches when every marker of `all`
## occurs in the lowercased license text and none of
## `unless` does; an entry matches when any of its sets
## does. The sets are written to be mutually exclusive:
## the `unless` markers keep multi-license texts on the
## entry carrying the stronger reference (a MIT-named
## file also quoting the GPL or the Apache license is a
## GPL/Apache file) and keep GPL sets off LGPL texts,
## whose name contains theirs. The entry order below
## (menu order) therefore only breaks ties.
##
## detect_files (top level): the candidate license file names the sniff
## reads, in preference order, shared by every license (the case-variant
## directory scan around them stays in Rust).
##
## license_url_template: the SPDX license page URL, with {id} substituted
## for the debian/copyright reference paragraph.
##
## The behavioral lock for the markers is the LICENSE_TEXTS test table in
## src/new/detect.rs: a bad marker edit fails those tests, not packages.
license_url_template: https://spdx.org/licenses/{id}.html
detect_files:
- LICENSE
- LICENSE.md
- LICENSE.txt
- COPYING
- COPYING.txt
# Entries in wizard-menu order.
licenses:
- id: MIT
menu: MIT
spellings: [MIT]
detect_markers:
- all:
- mit license
unless:
- apache license
- general public license
- all:
- permission is hereby granted, free of charge
unless:
- apache license
- general public license
- id: Apache-2.0
menu: Apache-2.0
spellings: [Apache-2.0]
detect_markers:
- all:
- apache license
- version 2
- id: GPL-2.0+
menu: GPL-2.0+
spellings: [GPL-2.0+]
detect_markers:
- all:
- general public license
unless:
- version 3
- lesser general public license
- id: GPL-3.0+
menu: GPL-3.0+
spellings: [GPL-3.0+]
detect_markers:
- all:
- general public license
- version 3
unless:
- lesser general public license
- id: LGPL-2.1+
menu: LGPL-2.1+
spellings: [LGPL-2.1+]
detect_markers:
- all:
- lesser general public license
unless:
- version 3
- all:
- lesser general public license
- version 2.1
- id: LGPL-3.0+
menu: LGPL-3.0+
spellings: [LGPL-3.0+]
detect_markers:
- all:
- lesser general public license
- version 3
unless:
- version 2.1
- id: BSD-2-Clause
menu: BSD-2-Clause
spellings: [BSD-2-Clause]
detect_markers:
- all:
- redistribution and use in source and binary forms
unless:
- endorse or promote
- isc license
- permission to use, copy, modify, and/or distribute this software
- id: BSD-3-Clause
menu: BSD-3-Clause
spellings: [BSD-3-Clause]
detect_markers:
- all:
- redistribution and use in source and binary forms
- endorse or promote
unless:
- isc license
- permission to use, copy, modify, and/or distribute this software
- id: ISC
menu: ISC
spellings: [ISC]
detect_markers:
- all:
- isc license
- all:
- permission to use, copy, modify, and/or distribute this software
+47
View File
@@ -0,0 +1,47 @@
# Quirks configuration for package-specific workarounds
# This file defines package-specific quirks that are applied during pull and deb operations
#
# `pull` and `deb` hold one entry per scope: several entries can carry
# different `series` lists, and every matching entry applies in file
# order. Entries can be scoped with `series`: an empty list applies to
# every series, otherwise only the listed ones. Packaging workarounds
# should carry the series they were verified against so they can be
# dropped once the upstream packaging catches up.
quirks:
# The resolute kernels declare `llvm-21-dev` unqualified while their
# other llvm pieces are `:native`; the dpkg cross rules then resolve it
# against the host architecture, whose dependency closure conflicts with
# the `:native` python3. Resolve it against the build architecture
# until the control is fixed upstream.
linux:
deb:
- series: [resolute]
dependencies:
replace:
llvm-21-dev: llvm-21-dev:native <!stage1>
linux-riscv:
deb:
- series: [resolute]
dependencies:
replace:
llvm-21-dev: llvm-21-dev:native <!stage1>
# Add more packages and their quirks as needed
# example-package:
# pull:
# - series: [noble]
# package_directory:
# - linux-main
# deb:
# - series: [resolute]
# dependencies:
# replace:
# llvm-21-dev: llvm-21-dev:native <!stage1>
# - series: [stonking]
# dependencies:
# replace:
# llvm-22-dev: llvm-22-dev:native <!stage1>
# parameters:
# key: value
+130
View File
@@ -0,0 +1,130 @@
---
name: pkh
description: 'Drive pkh, a Debian/Ubuntu packaging helper: pull source packages, generate changelog entries, build .dsc/.deb, lint, and upload to a PPA. Use it whenever the task touches Debian or Ubuntu packaging: patching an existing package, preparing an SRU, backport or NMU, scaffolding a new .deb, rebuilding for a PPA, or uploading a source package. Trigger on "update the changelog", "package this", or a bare package name, even when the user never mentions Debian.'
---
# pkh
`pkh` wraps the Debian packaging toolchain (`dch`, `dpkg-buildpackage`,
`sbuild`, `dpkg-source`, `quilt`, `lintian`, PPA uploads) in one CLI.
The subcommands share one set of option names, so `-s` always targets
the series and `--ppa` always names the PPA. Each step also does more
than the raw tool it replaces: `pull` fetches the orig tarball with the
source, `chlog` commits the entry it writes, `deb` sets up a chroot and
installs the build dependencies.
Check the install with `pkh --version`. Each command lists its flags
with `pkh <command> --help`, so check there instead of guessing. pkh
shells out to host tools (git, dpkg-dev, quilt, mmdebstrap, lintian,
pristine-tar, schroot, ...). Install the ones your workflow uses, or
use the classic snap, which carries them.
## Shared options
| Option | Meaning |
|---|---|
| `-d, --dist <dist>` | Target distribution, `debian` or `ubuntu` |
| `-s, --series <series>` | Target series, for example `resolute` or `noble` |
| `-v, --version <version>` | Target package version |
| `-a, --arch <arch>` | Target architecture, for example `amd64` or `riscv64` |
| `-p, --pocket <pocket>` | Distribution pocket: `updates`, `security`, `proposed` |
| `--ppa <user/ppa>` | Act on the named PPA |
Defaults come from the host vendor, its development series, and its
architecture. When the target differs, pass the flags: packaging for
Ubuntu on a Debian host needs `-d ubuntu`, and a series or architecture
that differs from the host needs `-s` or `-a`.
## Patch an Ubuntu package
```
pkh pull hello # source and orig tarball; add -d ubuntu off an Ubuntu host
# edit the package, committing each patch to git
pkh chlog # generates the entry, opens it for editing, commits it
git add debian/changelog && git commit -m "d/changelog"
pkh build # source package, written next to the tree
pkh deb # binary build in a chroot with build deps installed
pkh lint # lintian plus pkh-native checks
pkh put --ppa user/hello_xxx # uploads the .changes file from the build
git push xxx user-fork # push the branch to your fork
```
Run `pkh chlog` and `pkh build` from the root of the source tree; they
act on the package in the current directory.
## Command reference
- `pkh new [name]` scaffolds a buildable source package. `--lang`
picks the build system (`rust`, `python`, `meson`, `cmake`,
`autotools`, `go`, `shell`, `makefile`); `--source <PATH>` packages
existing sources instead. `--upstream-version` and `--revision` set
the version. `--description`, `--homepage`, `--license <SPDX>`,
`--command`, `--maintainer "Name <email>"`, and `--depends` fill in
the package metadata, with the maintainer defaulting to
`DEBFULLNAME`/`DEBEMAIL` and then git config. `--quilt` and
`--native` choose the source format; `--orig-from
release|git|path|snapshot` and `--orig-path` control the orig
tarball. The changelog starts as `UNRELEASED`; `--release` targets
`--series` instead. `--defaults` answers every remaining question
with its default, which keeps the run non-interactive.
- `pkh pull <package>` fetches a source package from the archive or
git. `--archive` skips git. `--ppa user/ppa` and `--repository
<suite-url>` pull from a PPA or an external flat repository instead.
`-d`, `-s`, `-v`, and `-p` target an exact source.
- `pkh chlog` generates the changelog entry from the commits since the
last version tag, opens it for editing, and commits it. `--backport`,
`--nmu`, and `--rebuild` apply the matching numbering scheme
(`3.1-1ubuntu2~24.04.1`, `1.0-1.1`, `1.0-1build1`); `-v` sets an
explicit version instead.
- `pkh build` produces the .dsc. `--orig auto|always|never` controls
whether the upload includes the orig tarball; the default, `auto`,
includes it only when the upstream version changed.
- `pkh deb` builds the binary packages in an isolated context with the
build dependencies installed. `--ppa` (repeatable) adds dependency
sources, `--inject <package|.deb>` preinstalls a package, and `-j`
caps parallel jobs. `--cross` cross-compiles instead of using
qemu-binfmt, but most packages cannot cross-compile, so prefer qemu.
Leave `--mode` unset unless you need a specific build context.
- `pkh lint [path]` runs lintian plus the pkh-native checks. `--json`
emits a machine-readable report and `--list-tags` prints the native
tag catalog. `--fail-on` sets the severities that fail the run
(errors by default), `--suppress-tags` ignores tags, `--check` runs a
single native check, and `--info`, `--pedantic`, and
`--experimental` add detail. `--repack` packs the tree fresh instead
of reusing the existing build output.
- `pkh put [changes]` uploads a .changes file to `--ppa user/ppa`.
With no argument it uploads the .changes from this package's last
build, found next to the source tree. `--force` re-uploads a file
that was already uploaded.
- `pkh prune` removes build artifacts and caches. Run it with
`--dry-run` first to list them. `--all` also deletes the cached
chroot tarballs, which take long to download again, so use it when
you need the disk space.
## Notes for agent runs
- Pass `-d`, `-s`, and `-a` whenever the target differs from the host,
so runs are reproducible.
- Keep runs non-interactive. Pass explicit flags, use `pkh new
--defaults`, and set `EDITOR` before `pkh chlog` (`EDITOR=true` keeps
the generated text). Commands may ask short questions on the
terminal; flags avoid most prompts.
- Pass `RUST_LOG=debug` for pkh's own logs. `--verbose` on `pkh build`
and `pkh deb` prints raw tool output instead of the live view.
- pkh writes the build artifacts (.dsc, .changes, logs) next to the
source tree. `pkh put` finds them without arguments, and `pkh prune`
removes them again.
- Run `pkh lint` before `pkh put`. It exits nonzero when findings reach
the `--fail-on` level, which defaults to errors.
- pkh intercepts Ctrl+C, runs its cleanup hooks, and exits with status
130. `pkh prune` removes anything left over.
The upstream repository is https://git.vhaudiquet.fr/vhaudiquet/pkh.
Its README has longer workflow examples.
+2
View File
@@ -0,0 +1,2 @@
bin_PROGRAMS = {command}
{command}_SOURCES = hello.c
@@ -0,0 +1,5 @@
AC_INIT([{name}], [{upstream_version}])
AM_INIT_AUTOMAKE([foreign])
AC_PROG_CC
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
+8
View File
@@ -0,0 +1,8 @@
#include <stdio.h>
/* Placeholder for {name}, generated by `pkh new`. */
int main(void)
{
printf("Hello from {command}!\n");
return 0;
}
+30
View File
@@ -0,0 +1,30 @@
## The `autotools` template: a C project with a configure.ac built through
## debhelper's auto-detection (dh runs autoreconf itself when it finds
## configure.ac, debhelper >= 10 — no override needed). The skeleton bodies
## below are static data (the first source build runs `autoreconf`,
## integrated in the dh sequence, so no generated configure script is
## committed); the logic half — the AC_INIT probe and the GNU-gettext
## detection (appended to Build-Depends) — lives in
## src/new/templates/autotools.rs. hello.c.tpl duplicates the shared C
## skeleton of the other C/C++ template directories (see
## meson/manifest.yml for why).
##
## Schema: see src/new/templates/mod.rs.
id: autotools
label: C/C++ (Autotools)
detect:
files: [configure.ac]
build_depends:
- autoconf
- automake
- libtool
architecture: any
rules_dh_line: "dh $@"
files:
- path: configure.ac
template: configure.ac.tpl
- path: Makefile.am
template: Makefile.am.tpl
- path: hello.c
template: hello.c.tpl
+5
View File
@@ -0,0 +1,5 @@
cmake_minimum_required(VERSION 3.16)
project({name} VERSION {upstream_version})
add_executable({command} hello.c)
install(TARGETS {command} RUNTIME DESTINATION bin)
+8
View File
@@ -0,0 +1,8 @@
#include <stdio.h>
/* Placeholder for {name}, generated by `pkh new`. */
int main(void)
{
printf("Hello from {command}!\n");
return 0;
}
+22
View File
@@ -0,0 +1,22 @@
## The `cmake` template: a C/C++ project built with CMake through the
## debhelper cmake buildsystem. The skeleton bodies below are static data;
## the logic half — the project() probe and the wizard's pkg-config
## opt-in (appended to Build-Depends) — lives in src/new/templates/cmake.rs.
## hello.c.tpl duplicates the shared C skeleton of the other C/C++
## template directories (see meson/manifest.yml for why).
##
## Schema: see src/new/templates/mod.rs.
id: cmake
label: C/C++ (CMake)
detect:
files: [CMakeLists.txt]
build_depends:
- cmake
architecture: any
rules_dh_line: "dh $@ --buildsystem=cmake"
files:
- path: CMakeLists.txt
template: CMakeLists.txt.tpl
- path: hello.c
template: hello.c.tpl
+1
View File
@@ -0,0 +1 @@
{name} - empty base tree scaffolded by `pkh new`; there is intentionally no upstream build system here.
+19
View File
@@ -0,0 +1,19 @@
## The `empty` template: a metapackage (non-empty Depends list) or an
## empty base package with no build system at all — pure `dh $@` plumbing
## as a starting point for hand-written rules. Pure data: no hooks, the
## metapackage Depends payload travels in the wizard answers, and the
## only upstream file is the stub README marking the tree as
## intentionally empty.
##
## Schema: see src/new/templates/mod.rs.
id: empty
label: Metapackage / empty base (no build system)
detect:
files: []
build_depends: []
architecture: all
rules_dh_line: "dh $@"
files:
- path: README
template: README.tpl
+3
View File
@@ -0,0 +1,3 @@
module {name}
go 1.21
+8
View File
@@ -0,0 +1,8 @@
// Placeholder for {name}, generated by `pkh new`.
package main
import "fmt"
func main() {
fmt.Println("Hello from {command}!")
}
+24
View File
@@ -0,0 +1,24 @@
## The `go` template: a Go module built through dh-golang. The logic half
## — the go.mod module-line probe and the `{go_import_path}` value below —
## lives in src/new/templates/go.rs; the skeleton bodies are static data
## (the `go` directive of go.mod stays a literal: nothing about it is
## answer-derived, so it has no {placeholder}).
##
## Schema: see src/new/templates/mod.rs.
id: go
label: Go module
detect:
files: [go.mod]
build_depends:
- golang-any
- dh-golang
architecture: any
rules_dh_line: "dh $@ --buildsystem=golang"
source_fields:
XS-Go-Import-Path: "{go_import_path}"
files:
- path: go.mod
template: go.mod.tpl
- path: main.go
template: main.go.tpl
+16
View File
@@ -0,0 +1,16 @@
CC ?= cc
CFLAGS ?= -O2 -Wall -Wextra
PREFIX ?= /usr
all: {command}
{command}: hello.c
$(CC) $(CFLAGS) -o $@ hello.c
install: {command}
install -Dm755 {command} $(DESTDIR)$(PREFIX)/bin/{command}
clean:
rm -f {command}
.PHONY: all install clean
+8
View File
@@ -0,0 +1,8 @@
#include <stdio.h>
/* Placeholder for {name}, generated by `pkh new`. */
int main(void)
{
printf("Hello from {command}!\n");
return 0;
}
+1
View File
@@ -0,0 +1 @@
{command} usr/bin/{command}
+28
View File
@@ -0,0 +1,28 @@
## The `makefile` template: a generic project driven by a plain Makefile.
## debhelper's makefile buildsystem runs `make` for the build and
## `make install DESTDIR=...` when the Makefile carries an `install:`
## target (missing targets are skipped gracefully), so plain `dh $@`
## plumbing is enough here. The phony-install hint of
## src/new/templates/makefile.rs (whether dh_auto_install will run
## `make install` for an existing tree) is the only logic; the skeleton
## bodies below are static data (the install mapping is rendered for
## skeletons only, whose phony install target is known by construction).
##
## Schema: see src/new/templates/mod.rs.
id: makefile
label: Generic (Makefile)
detect:
files: [Makefile]
build_depends:
- build-essential
architecture: any
rules_dh_line: "dh $@"
files:
- path: hello.c
template: hello.c.tpl
- path: Makefile
template: Makefile.tpl
- path: debian/install
template: install.tpl
skeleton_only: true
+8
View File
@@ -0,0 +1,8 @@
#include <stdio.h>
/* Placeholder for {name}, generated by `pkh new`. */
int main(void)
{
printf("Hello from {command}!\n");
return 0;
}
+28
View File
@@ -0,0 +1,28 @@
## The `meson` template: a C/C++ project built with Meson through the
## debhelper meson buildsystem. The skeleton bodies below are static data;
## the logic half — the project() probe and the wizard's pkg-config
## opt-in (appended to Build-Depends) — lives in src/new/templates/meson.rs.
##
## hello.c.tpl is deliberately duplicated (byte-identical) across the
## makefile, cmake and autotools template directories: every template
## directory is self-contained — the registry embeds each directory's
## bodies under its own entry — so a shared body would need
## cross-directory references the manifest schema has no machinery for.
## The duplication replaces the Rust hello_c() helper meson.rs used to
## lend cmake.rs and autotools.rs.
##
## Schema: see src/new/templates/mod.rs.
id: meson
label: C/C++ (Meson)
detect:
files: [meson.build]
build_depends:
- meson
architecture: any
rules_dh_line: "dh $@ --buildsystem=meson"
files:
- path: meson.build
template: meson.build.tpl
- path: hello.c
template: hello.c.tpl
+3
View File
@@ -0,0 +1,3 @@
project('{name}', version: '{upstream_version}', license: '{license}', default_options: ['c_std=c11'])
executable('{command}', 'hello.c', install: true)
+5
View File
@@ -0,0 +1,5 @@
"""Placeholder for {name}, generated by `pkh new`."""
def main() -> None:
print("Hello from {command}!")
+29
View File
@@ -0,0 +1,29 @@
## The `python` template: a PEP 517 project built with pybuild. The
## skeleton bodies below are static data on the fresh-skeleton baseline
## (the setuptools backend): the module directory and the console-script
## entry point are named by the `{module_name}` placeholder python.rs
## derives from the package name — dpkg names may carry `+`/`.` and may
## start with a digit, none of which a Python module name may. The logic
## half — the pyproject.toml/setup.py probe and the Build-Depends /
## architecture resolution for existing projects (backend package,
## pyproject presence, C-extension hints) — lives in
## src/new/templates/python.rs.
##
## Schema: see src/new/templates/mod.rs.
id: python
label: Python (pyproject.toml / setup.py)
detect:
files: [pyproject.toml, setup.py, setup.cfg]
build_depends:
- dh-python
- python3-all
- pybuild-plugin-pyproject
- python3-setuptools
architecture: all
rules_dh_line: "dh $@ --with python3 --buildsystem=pybuild"
files:
- path: pyproject.toml
template: pyproject.toml.tpl
- path: "{module_name}/__init__.py"
template: __init__.py.tpl
+12
View File
@@ -0,0 +1,12 @@
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "{name}"
version = "{upstream_version}"
description = "{summary}"
requires-python = ">=3.8"
[project.scripts]
{command} = "{module_name}:main"
+6
View File
@@ -0,0 +1,6 @@
[package]
name = "{crate_name}"
version = "{upstream_version}"
edition = "2021"
[dependencies]
+4
View File
@@ -0,0 +1,4 @@
// Placeholder for {name}, generated by `pkh new`.
fn main() {
println!("Hello from {command}!");
}
+33
View File
@@ -0,0 +1,33 @@
## The `rust` template: a vendored Cargo build (see the module docs of
## src/new/templates/rust.rs for the vendoring strategy). The skeleton
## bodies and the vendored-build rules overrides below are static data; the
## logic half — the cargo vendor post-write hook, the project probe, and
## the `{crate_name}` / `{locked}` / `{artifact}` values of the bodies —
## lives in that module (dpkg package names may carry `+`/`.`, which cargo
## rejects in crate names, so the skeleton crate name is a derived
## placeholder, not the raw `{name}`).
##
## Schema: see src/new/templates/mod.rs.
id: rust
label: Rust (Cargo.toml)
detect:
files: [Cargo.toml]
build_depends:
- cargo:native
- rustc:native
architecture: any
rules_dh_line: "dh $@"
# The vendored-build overrides appended to debian/rules; `--locked` is only
# used when the packaged tree already carries a Cargo.lock (the vendoring
# hook patches it in once it creates the lockfile), and the built artifact
# of a fresh skeleton is named after its crate.
rules_extra_file: rules.extra.tpl
gitignore_entries:
- vendor/
- .cargo/config.toml
files:
- path: Cargo.toml
template: Cargo.toml.tpl
- path: src/main.rs
template: main.rs.tpl
+19
View File
@@ -0,0 +1,19 @@
override_dh_auto_build:
cargo build --release --offline{locked}
override_dh_auto_install:
install -Dm755 target/release/{artifact} debian/{name}/usr/bin/{command}
override_dh_auto_test:
cargo test --release --offline{locked}
override_dh_update_autotools_config:
override_dh_clean:
# dh_clean unlinks `*.orig` patch backups, but vendored crates
# ship files like `Cargo.toml.orig` that cargo's per-file
# checksums require on cold builds (chroots, Launchpad).
dh_clean -X .orig
override_dh_auto_clean:
cargo clean
+1
View File
@@ -0,0 +1 @@
{command}.sh usr/bin/{command}
+25
View File
@@ -0,0 +1,25 @@
## The `shell` template: a single interpreted script installed to
## /usr/bin with plain `dh $@` plumbing. Detection is not marker-based: the
## single-script heuristic of src/new/detect.rs (a lone *.sh or shebang
## file) maps here. The probe pre-filling the wizard answers from the
## script file name lives in src/new/templates/shell.rs; everything else
## is the data below (the skeleton script is executable, the install
## mapping exists for skeletons only — packaging an existing tree leaves
## the mapping to the user).
##
## Schema: see src/new/templates/mod.rs.
id: shell
label: Shell script / single interpreted file
detect:
files: []
build_depends: []
architecture: all
rules_dh_line: "dh $@"
files:
- path: "{command}.sh"
template: script.tpl
executable: true
- path: debian/install
template: install.tpl
skeleton_only: true
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
# Placeholder for {name}, generated by `pkh new`.
echo "Hello from {command}!"
-28
View File
@@ -1,28 +0,0 @@
## Static data needed for pkh operations
## Instead of hardcoding the data in code, data files allow to quickly
## update and maintain such data in one unique place
## The goal is to have the minimal possible set of data necessary
## to grab the actual data. For example we don't want to store every Ubuntu
## or Debian series, but rather an URL where we can properly access that data.
dist_info:
local: /usr/share/distro-info/{dist}
network: https://salsa.debian.org/debian/distro-info-data/-/raw/main/
dist:
debian:
base_url: http://deb.debian.org/debian
archive_keyring: https://ftp-master.debian.org/keys/archive-key-{series_num}.asc
pockets:
- proposed-updates
- updates
series:
local: /usr/share/distro-info/debian.csv
network: https://salsa.debian.org/debian/distro-info-data/-/raw/main/debian.csv
ubuntu:
base_url: http://archive.ubuntu.com/ubuntu
archive_keyring: http://archive.ubuntu.com/ubuntu/project/ubuntu-archive-keyring.gpg
pockets:
- proposed
- updates
series:
local: /usr/share/distro-info/ubuntu.csv
network: https://salsa.debian.org/debian/distro-info-data/-/raw/main/ubuntu.csv
-15
View File
@@ -1,15 +0,0 @@
# Quirks configuration for package-specific workarounds
# This file defines package-specific quirks that are applied during pull and deb operations
quirks:
# Add more packages and their quirks as needed
# example-package:
# pull:
# method: archive
# deb:
# extra_dependencies:
# - another-dependency
# parameters:
# key: value
+81 -8
View File
@@ -5,9 +5,14 @@ description: |
pkh aims at wrapping the different debian tools and workflows
into one tool, that would have the same interface for everything,
while being smarter at integrating all workflows.
This snap uses classic confinement and carries the packaging
toolchain it drives (dpkg-dev, git, mmdebstrap, lintian, quilt, ...)
so it behaves the same on any Debian/Ubuntu host.
license: MIT OR GPL-2.0-only
adopt-info: pkh-part
confinement: devmode
confinement: classic
apps:
pkh:
@@ -19,24 +24,92 @@ parts:
source: .
override-pull: |
craftctl default
craftctl set version=$(git rev-parse --short=11 HEAD)
craftctl set grade="devel"
# Release metadata comes from the crate, not the git state: a build
# of any commit must produce the version the crate declares.
craftctl set version="$(awk -F'"' '/^version =/{print $2; exit}' Cargo.toml)"
craftctl set grade="stable"
build-packages:
- build-essential
- file
- patchelf
- pkg-config
- libssl-dev
- libgpg-error-dev
- libgpgme-dev
# Host-side tools pkh execs directly. Tools that only run *inside*
# the build chroot (dose-builddebcheck, dpkg-cross) are provisioned
# there by pkh itself and must not be staged; likewise qemu-user-static
# is host binfmt configuration, not a bundled file.
#
# The apt and dpkg state-owning tools are deliberately excluded below:
# they must be the host's (classic mode makes them visible), since a
# core24 apt/dpkg managing a newer host's package database is exactly
# the version skew classic snaps must avoid. The source-package tools
# (dpkg-buildpackage, dpkg-source, ...) are bundled instead.
stage-packages:
- libgpgme11t64
- git
- curl
- gnupg
- gpgv
- dpkg-dev
- quilt
- pristine-tar
- mmdebstrap
- lintian
- fakeroot
- util-linux
- dpkg-dev
# mount/umount moved to their own package (split from util-linux)
- mount
- schroot
- openssh-client
- tar
- xz-utils
- bzip2
stage:
- -usr/lib/x86_64-linux-gnu/libicuio.so.74.2
- -usr/lib/x86_64-linux-gnu/libicutest.so.74.2
- -usr/lib/x86_64-linux-gnu/libicutu.so.74.2
- -usr/lib/x86_64-linux-gnu/libicui18n.so.74.2
- -usr/bin/apt
- -usr/bin/apt-cache
- -usr/bin/apt-cdrom
- -usr/bin/apt-config
- -usr/bin/apt-get
- -usr/bin/apt-key
- -usr/bin/apt-mark
- -usr/lib/*/libapt-*
- -usr/lib/*/libicuio*
- -usr/lib/*/libicutest*
- -usr/lib/*/libicutu*
- -usr/lib/*/libicui18n*
# update-alternatives does not run at staging time: expose the sysv
# fakeroot under the plain name dpkg-buildpackage and pkh exec.
override-prime: |
craftctl default
ln -sfn fakeroot-sysv "${CRAFT_PRIME}/usr/bin/fakeroot"
# Ship the license texts with the binary: the MIT grant requires
# the notice to accompany copies, and GPL-2 requires the license
# text alongside distribution.
mkdir -p "${CRAFT_PRIME}/usr/share/doc/pkh"
cp "${CRAFT_PART_SRC}/LICENSE-MIT" "${CRAFT_PART_SRC}/LICENSE-GPL" \
"${CRAFT_PRIME}/usr/share/doc/pkh/"
# Classic-confined ELFs default to the host loader, which pins the
# snap to hosts shipping at least the build environment's glibc,
# and cannot see the libraries deduplicated against the base.
# Point every bundled ELF at the core24 loader and give it an
# rpath resolving base libraries from the mounted base and
# snap-local libraries from $ORIGIN — the classic linter's
# guidance, and what Canonical's own classic snaps do. DT_RPATH
# (--force-rpath) is required over the default DT_RUNPATH: the
# host ld.so.cache would otherwise resolve sonames to host
# libraries first, mixing host libm/libresolv with base libc.
# DT_RPATH also propagates transitively, covering dependencies of
# dependencies (libgpgme -> libassuan). Host tools spawned later
# (host apt-get, ...) run with a pristine environment since no
# LD_LIBRARY_PATH is exported.
find "${CRAFT_PRIME}" -type f -exec sh -c '
for f do
[ "$(od -An -N4 -tx1 "$f" | tr -d " \n")" = "7f454c46" ] || continue
patchelf --set-interpreter \
/snap/core24/current/lib64/ld-linux-x86-64.so.2 "$f" 2>/dev/null || true
patchelf --force-rpath --set-rpath \
"/snap/core24/current/lib/x86_64-linux-gnu:/snap/core24/current/usr/lib/x86_64-linux-gnu:\$ORIGIN:\$ORIGIN/../lib/x86_64-linux-gnu:\$ORIGIN/../usr/lib/x86_64-linux-gnu" \
"$f" 2>/dev/null || true
done' sh {} +
+171 -37
View File
@@ -4,12 +4,34 @@
//! for mmdebstrap operations and for PPA packages by downloading them.
use crate::context;
use crate::data::embed_data;
use crate::distro_info;
use serde::Deserialize;
use std::error::Error;
use std::os::unix::fs::MetadataExt;
use std::path::{Path, PathBuf};
use std::sync::Arc;
/// Keyserver endpoint, loaded from the bundled `keyserver.yml` data file
/// (same pattern as `distro_info.yml`): the lookup URL is a static
/// endpoint that was previously hardcoded in two modules.
#[derive(Debug, Deserialize)]
struct KeyserverData {
/// OpenPGP key lookup URL template (`{fingerprint}`)
lookup_template: String,
}
embed_data! {
static ref KEYSERVER_DATA: KeyserverData = "../../data/keyserver.yml"
}
/// URL fetching the OpenPGP key of `fingerprint` from the keyserver
pub(crate) fn keyserver_lookup_url(fingerprint: &str) -> String {
KEYSERVER_DATA
.lookup_template
.replace("{fingerprint}", fingerprint)
}
/// Launchpad API response structure for PPA information
#[derive(Deserialize)]
struct LaunchpadPpaResponse {
@@ -47,18 +69,50 @@ pub async fn download_cache_keyrings(
// Use system temp directory for keyrings since it's accessible from unshare mode
// The home directory may not be accessible from mmdebstrap's unshare namespace
let temp_dir = std::env::temp_dir();
let keyring_dir = temp_dir.join("pkh-keyrings");
// Name the cache directory per-uid: a single shared /tmp directory would
// be writable by any local user, and the skip-if-exists logic below
// trusts pre-existing keyrings, so it must never be shared.
let euid = current_euid();
let keyring_dir = temp_dir.join(format!("pkh-keyrings-{euid}"));
// Create keyring directory if it doesn't exist
if !ctx.exists(&keyring_dir)? {
ctx.command("mkdir").arg("-p").arg(&keyring_dir).status()?;
if ctx.exists(&keyring_dir)? {
if let context::ContextConfig::Local = ctx.config {
// Cached keyrings are trusted as-is whenever they already exist,
// so refuse to reuse a directory that is not owned by the current
// user or is writable by group/others (it could have been planted
// by another local user).
let metadata = std::fs::symlink_metadata(&keyring_dir)?;
validate_keyring_dir(metadata.uid(), metadata.mode(), euid).map_err(|reason| {
format!(
"Refusing to use keyring cache directory {}: {reason}; \
remove the directory and re-run pkh",
keyring_dir.display()
)
})?;
// Upgrade cache directories created by versions that made them
// private: mmdebstrap's unshare-mode hooks cannot read them.
} else {
// Remote contexts (e.g. ssh) have no stat/metadata access through
// the context API, so the ownership guard cannot be performed;
// keep the previous best-effort behavior of tightening the
// directory permissions instead (no group/others write).
}
// Make keyring directory world-accessible so mmdebstrap in unshare mode can access it
ctx.command("chmod")
.arg("a+rwx")
ctx.command("chmod").arg("755").arg(&keyring_dir).status()?;
} else {
// Create the directory readable but not writable by group/others.
// mmdebstrap's unshare-mode hooks run under an identity that cannot
// read the invoking user's private directories, so 0700 breaks the
// keyring copy into the chroot; the planting guard stays on the
// ownership and no-write checks of validate_keyring_dir (the
// skip-if-exists logic below trusts pre-existing keyrings, so the
// directory must never be writable by anyone else).
ctx.command("mkdir")
.arg("-p")
.arg("-m")
.arg("755")
.arg(&keyring_dir)
.status()?;
}
for keyring_url in keyring_urls {
// Extract the original filename from the keyring URL
@@ -116,9 +170,6 @@ pub async fn download_cache_keyrings(
let _ = ctx.command("rm").arg("-f").arg(&download_path).status();
}
// Make the keyring file world-readable so mmdebstrap in unshare mode can access it
ctx.command("chmod").arg("a+r").arg(&binary_path).status()?;
log::info!(
"Successfully downloaded keyring for {} to {}",
series,
@@ -129,9 +180,12 @@ pub async fn download_cache_keyrings(
"Keyring already exists at {}, skipping download",
binary_path.display()
);
// Ensure existing keyring is world-readable
ctx.command("chmod").arg("a+r").arg(&binary_path).status()?;
}
// Readable like the directory: mmdebstrap's hooks copy these into
// the chroot. Applies to legacy files too, which a restrictive
// umask may have left private, and a permissive one group-writable.
let _ = ctx.command("chmod").arg("644").arg(&binary_path).status();
}
log::info!(
@@ -143,7 +197,42 @@ pub async fn download_cache_keyrings(
Ok(keyring_dir)
}
/// Download and import a PPA key using Launchpad API
/// Effective uid of the current process
fn current_euid() -> u32 {
unsafe { libc::geteuid() }
}
/// Check that an existing keyring cache directory is safe to reuse
///
/// Cached keyrings are trusted whenever the files already exist (see the
/// skip-if-exists logic in [`download_cache_keyrings`]), so the directory
/// must be owned by the current user and must not be writable by group or
/// others, otherwise another local user could plant a malicious keyring.
///
/// Takes the directory's owner uid and permission mode (e.g. from
/// `std::fs::symlink_metadata`) so it can be unit tested without touching
/// the filesystem.
fn validate_keyring_dir(dir_uid: u32, mode: u32, euid: u32) -> Result<(), String> {
if dir_uid != euid {
return Err(format!(
"owned by uid {dir_uid}, not by the current user (uid {euid})"
));
}
if mode & 0o022 != 0 {
return Err(format!(
"writable by group or others (permissions {:04o})",
mode & 0o7777
));
}
Ok(())
}
/// Download and import a PPA key using the Launchpad API
///
/// The signing key fingerprint is looked up through the shared HTTP client;
/// the key itself is fetched from the keyserver with curl through the
/// context, because the key file must land in the context's filesystem
/// (which may be remote).
///
/// # Arguments
/// * `ctx` - Optional context to use
@@ -174,33 +263,36 @@ pub async fn download_trust_ppa_key(
ppa_name
);
// Get PPA information from Launchpad API to get signing key fingerprint
// Use the correct devel API endpoint
let api_url = format!(
"https://api.launchpad.net/1.0/~{}/+archive/ubuntu/{}",
ppa_owner, ppa_name
);
// Get PPA information from the Launchpad API to get the signing key
// fingerprint. The query is context-independent metadata, so it goes
// through the shared HTTP client (timeouts, retries) rather than
// shelling out to curl.
let api_url = crate::launchpad::archive_url(ppa_owner, ppa_name);
log::debug!("Querying Launchpad API: {}", api_url);
let api_response = ctx
.command("curl")
.arg("-s")
.arg("-f")
.arg("-H")
.arg("Accept: application/json")
.arg(&api_url)
.output()?;
if !api_response.status.success() {
let response = distro_info::http_get_retried(&api_url).await.map_err(|e| {
format!(
"Failed to query Launchpad API for PPA {}/{}: {}",
ppa_owner, ppa_name, e
)
})?;
if !response.status().is_success() {
return Err(format!(
"Failed to query Launchpad API for PPA {}/{}",
ppa_owner, ppa_name
"Failed to query Launchpad API for PPA {}/{}: HTTP {}",
ppa_owner,
ppa_name,
response.status()
)
.into());
}
// Parse the JSON response to extract the signing key fingerprint
let api_response_str = String::from_utf8_lossy(&api_response.stdout);
let api_response_str = response.text().await.map_err(|e| {
format!(
"Failed to read the Launchpad API response for PPA {}/{}: {}",
ppa_owner, ppa_name, e
)
})?;
let ppa_response: LaunchpadPpaResponse =
serde_json::from_str(&api_response_str).map_err(|e| {
format!(
@@ -213,10 +305,7 @@ pub async fn download_trust_ppa_key(
log::debug!("Found PPA signing key fingerprint: {}", fingerprint);
// Download the actual key from the keyserver using the fingerprint
let keyserver_url = format!(
"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x{}",
fingerprint
);
let keyserver_url = keyserver_lookup_url(&fingerprint);
log::debug!("Downloading key from keyserver: {}", keyserver_url);
let mut curl_cmd = ctx.command("curl");
@@ -246,3 +335,48 @@ pub async fn download_trust_ppa_key(
);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
/// The data-driven template renders the lookup URL the hardcoded
/// format! used to build (verified against the live keyserver)
#[test]
fn keyserver_lookup_url_substitutes_the_fingerprint() {
assert_eq!(
keyserver_lookup_url("0123456789ABCDEF"),
"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x0123456789ABCDEF"
);
}
#[test]
fn test_validate_keyring_dir_accepts_private_dir_owned_by_current_user() {
assert!(validate_keyring_dir(1000, 0o700, 1000).is_ok());
assert!(validate_keyring_dir(1000, 0o750, 1000).is_ok());
assert!(validate_keyring_dir(1000, 0o1744, 1000).is_ok());
assert!(validate_keyring_dir(0, 0o700, 0).is_ok());
// The world-readable modes the cache now uses: readable so that
// mmdebstrap's unshare-mode hooks can copy the keyrings, while the
// ownership and no-write checks keep the planting guard.
assert!(validate_keyring_dir(1000, 0o755, 1000).is_ok());
}
#[test]
fn test_validate_keyring_dir_rejects_foreign_owner() {
let err = validate_keyring_dir(1000, 0o700, 1001).unwrap_err();
assert!(err.contains("owned by uid 1000"));
let err = validate_keyring_dir(1001, 0o700, 1000).unwrap_err();
assert!(err.contains("owned by uid 1001"));
}
#[test]
fn test_validate_keyring_dir_rejects_group_or_other_writable() {
assert!(validate_keyring_dir(1000, 0o770, 1000).is_err());
assert!(validate_keyring_dir(1000, 0o706, 1000).is_err());
assert!(validate_keyring_dir(1000, 0o707, 1000).is_err());
assert!(validate_keyring_dir(1000, 0o777, 1000).is_err());
// Sticky bit does not neutralize the group/other write bits.
assert!(validate_keyring_dir(1000, 0o1777, 1000).is_err());
}
}
+2
View File
@@ -1,2 +1,4 @@
pub mod keyring;
/// Release-file signature and checksum verification for repositories
pub mod release;
pub mod sources;
+1303
View File
File diff suppressed because it is too large Load Diff
+544 -115
View File
@@ -1,97 +1,188 @@
//! APT sources.list management
//! Provides a simple structure for managing APT repository sources
use crate::context;
//!
//! Entries carry enough information (kind, signed-by, trusted, enabled) to
//! be written back without loss, and remember the file they were loaded
//! from ([`SourceEntry::origin`]) so that saving writes each entry back to
//! its own file, in that file's own format.
use crate::context::{self, Context};
use crate::debian::control::{Paragraph, parse_paragraphs, write_paragraph};
use std::error::Error;
use std::path::Path;
use std::path::{Path, PathBuf};
use std::sync::Arc;
/// Sources file owned by pkh, holding entries added by pkh (e.g. PPAs).
///
/// New entries never end up in distro-managed files.
const PKH_ADDED_PATH: &str = "/etc/apt/sources.list.d/pkh-added.list";
/// Suffix appended to an origin file path to build its backup path
const BACKUP_SUFFIX: &str = ".pkh-backup";
/// Kind of packages provided by a source entry
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum SourceKind {
/// Binary packages ('deb')
Deb,
/// Source packages ('deb-src')
DebSrc,
}
impl SourceKind {
/// Token used in legacy lines and deb822 'Types' fields
pub fn as_str(self) -> &'static str {
match self {
SourceKind::Deb => "deb",
SourceKind::DebSrc => "deb-src",
}
}
/// Parse a type token ('deb' or 'deb-src')
fn parse(token: &str) -> Option<Self> {
match token {
"deb" => Some(SourceKind::Deb),
"deb-src" => Some(SourceKind::DebSrc),
_ => None,
}
}
}
/// On-disk format of a sources file
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum SourceFormat {
/// Legacy one-line-per-entry format (sources.list, *.list)
Legacy,
/// deb822 format (*.sources)
Deb822,
}
/// File a source entry was loaded from
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SourceOrigin {
/// Path of the origin file, inside the context
pub path: PathBuf,
/// Format of the origin file
pub format: SourceFormat,
}
/// Represents a single source entry in sources.list
#[derive(Debug, Clone)]
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SourceEntry {
/// Is the source enabled?
pub enabled: bool,
/// Kind of packages provided by the source (binary or source)
pub kind: SourceKind,
/// Source components (universe, main, contrib)
pub components: Vec<String>,
/// Source architectures (amd64, riscv64, arm64)
pub architectures: Vec<String>,
/// Keyring the repository is signed with ('signed-by' option)
pub signed_by: Option<String>,
/// Explicit trust flag ('trusted' option), when set
pub trusted: Option<bool>,
/// Source URI
pub uri: String,
/// Source suites (series-pocket)
pub suite: Vec<String>,
/// File and format the entry was loaded from
///
/// Entries without an origin are new (e.g. repositories added by pkh);
/// they are saved to the pkh-owned added-sources file.
pub origin: Option<SourceOrigin>,
}
impl SourceEntry {
/// Parse a string describing a source entry in deb822 format
pub fn from_deb822(data: &str) -> Option<Self> {
let mut current_entry = SourceEntry {
enabled: true,
components: Vec::new(),
architectures: Vec::new(),
uri: String::new(),
suite: Vec::new(),
};
for line in data.lines() {
let line = line.trim();
if line.starts_with('#') {
continue;
/// Build entries from a single deb822 stanza
///
/// A stanza declaring several types ('Types: deb deb-src') yields one
/// entry per type.
fn from_deb822_stanza(p: &Paragraph) -> Vec<Self> {
// apt defaults 'Types' to 'deb' when the field is absent
let mut kinds: Vec<SourceKind> = p
.get("Types")
.unwrap_or("deb")
.split_whitespace()
.filter_map(SourceKind::parse)
.collect();
if kinds.is_empty() {
kinds.push(SourceKind::Deb);
}
// Empty line: end of an entry, or beginning
if line.is_empty() {
if !current_entry.uri.is_empty() {
return Some(current_entry);
} else {
continue;
}
let enabled = p
.get("Enabled")
.map(|v| {
let v = v.trim();
!v.eq_ignore_ascii_case("no") && !v.eq_ignore_ascii_case("false")
})
.unwrap_or(true);
let signed_by = p
.get("Signed-By")
.map(str::trim)
.filter(|v| !v.is_empty())
.map(str::to_string);
let trusted = p
.get("Trusted")
.map(|v| v.trim().eq_ignore_ascii_case("yes"));
let uri = p.get("URIs").unwrap_or("").trim().to_string();
if uri.is_empty() {
return Vec::new();
}
let suite: Vec<String> = p
.get("Suites")
.unwrap_or("")
.split_whitespace()
.map(|s| s.to_string())
.collect();
let components: Vec<String> = p
.get("Components")
.unwrap_or("")
.split_whitespace()
.map(|s| s.to_string())
.collect();
let architectures: Vec<String> = p
.get("Architectures")
.unwrap_or("")
.split_whitespace()
.map(|s| s.to_string())
.collect();
if let Some((key, value)) = line.split_once(':') {
let key = key.trim();
let value = value.trim();
match key {
"Types" => {
// We only care about deb types
}
"URIs" => current_entry.uri = value.to_string(),
"Suites" => {
current_entry.suite =
value.split_whitespace().map(|s| s.to_string()).collect();
}
"Components" => {
current_entry.components =
value.split_whitespace().map(|s| s.to_string()).collect();
}
"Architectures" => {
current_entry.architectures =
value.split_whitespace().map(|s| s.to_string()).collect();
}
_ => {}
}
}
}
// End of entry, or empty file?
if !current_entry.uri.is_empty() {
Some(current_entry)
} else {
None
}
kinds
.into_iter()
.map(|kind| SourceEntry {
enabled,
kind,
components: components.clone(),
architectures: architectures.clone(),
signed_by: signed_by.clone(),
trusted,
uri: uri.clone(),
suite: suite.clone(),
origin: None,
})
.collect()
}
/// Parse a line describing a legacy source entry
pub fn from_legacy(data: &str) -> Option<Self> {
let line = data.lines().next()?.trim();
let raw = data.lines().next()?.trim();
if line.is_empty() || line.starts_with("#") {
if raw.is_empty() {
return None;
}
// Parse legacy deb line format: deb [arch=... / signed_by=] uri suite [components...]
// Entries commented out with '#' are disabled, not deleted
let (enabled, line) = match raw.strip_prefix('#') {
Some(rest) => (false, rest.trim_start()),
None => (true, raw),
};
// Parse legacy deb line format:
// deb [arch=... signed-by=... trusted=...] uri suite [components...]
// Extract bracket parameters first
let mut architectures = Vec::new();
let mut signed_by = None;
let mut trusted = None;
let mut line_without_brackets = line.to_string();
// Find and process bracket parameters
@@ -102,14 +193,13 @@ impl SourceEntry {
// Parse parameters inside brackets
for param in bracket_content.split_whitespace() {
if param.starts_with("arch=") {
let arch_values = param.split('=').nth(1).unwrap_or("");
architectures = arch_values
.split(',')
.map(|s| s.trim().to_string())
.collect();
if let Some(values) = param.strip_prefix("arch=") {
architectures = values.split(',').map(|s| s.trim().to_string()).collect();
} else if let Some(keyring) = param.strip_prefix("signed-by=") {
signed_by = Some(keyring.trim_matches('"').to_string());
} else if let Some(flag) = param.strip_prefix("trusted=") {
trusted = Some(flag.eq_ignore_ascii_case("yes") || flag == "1");
}
// signed-by parameter is parsed but not stored
}
// Remove the bracket section from the line
@@ -120,37 +210,61 @@ impl SourceEntry {
let line_without_brackets = line_without_brackets.trim();
let parts: Vec<&str> = line_without_brackets.split_whitespace().collect();
// We need at least: deb, uri, suite
if parts.len() < 3 || parts[0] != "deb" {
// We need at least: type, uri, suite
if parts.len() < 3 {
return None;
}
let kind = SourceKind::parse(parts[0])?;
let uri = parts[1].to_string();
let suite = vec![parts[2].to_string()];
let components: Vec<String> = parts[3..].iter().map(|&s| s.to_string()).collect();
Some(SourceEntry {
enabled: true,
enabled,
kind,
components,
architectures,
signed_by,
trusted,
uri,
suite,
origin: None,
})
}
/// Convert this source entry to legacy format
///
/// Entries holding several suites are rendered as one line per suite.
/// Disabled entries are commented out.
pub fn to_legacy(&self) -> String {
let mut result = String::new();
// Legacy entries contain one suite per line
for suite in &self.suite {
// Start with "deb" type
result.push_str("deb");
if !self.enabled {
result.push_str("# ");
}
result.push_str(self.kind.as_str());
// Add architectures if present
// Bracket options: architectures, signing keyring and trust
let mut options = Vec::new();
if !self.architectures.is_empty() {
result.push_str(" [arch=");
result.push_str(&self.architectures.join(","));
options.push(format!("arch={}", self.architectures.join(",")));
}
if let Some(keyring) = &self.signed_by {
if keyring.contains(char::is_whitespace) {
options.push(format!("signed-by=\"{keyring}\""));
} else {
options.push(format!("signed-by={keyring}"));
}
}
if let Some(trusted) = self.trusted {
options.push(format!("trusted={}", if trusted { "yes" } else { "no" }));
}
if !options.is_empty() {
result.push_str(" [");
result.push_str(&options.join(" "));
result.push(']');
}
@@ -171,88 +285,199 @@ impl SourceEntry {
result
}
/// Convert this source entry to a deb822 stanza (with a trailing newline)
pub fn to_deb822(&self) -> String {
let mut stanza = Paragraph::new();
stanza.set("Types", self.kind.as_str());
stanza.set("URIs", &self.uri);
stanza.set("Suites", &self.suite.join(" "));
stanza.set("Components", &self.components.join(" "));
if let Some(keyring) = &self.signed_by {
stanza.set("Signed-By", keyring);
}
if !self.architectures.is_empty() {
stanza.set("Architectures", &self.architectures.join(" "));
}
if let Some(trusted) = self.trusted {
stanza.set("Trusted", if trusted { "yes" } else { "no" });
}
if !self.enabled {
stanza.set("Enabled", "no");
}
write_paragraph(&stanza)
}
}
/// Parse a 'source list' string in deb822 format into a SourceEntry vector
///
/// A stanza declaring several types ('Types: deb deb-src') yields one entry
/// per type.
pub fn parse_deb822(data: &str) -> Vec<SourceEntry> {
data.split("\n\n")
.flat_map(SourceEntry::from_deb822)
parse_paragraphs(data)
.iter()
.flat_map(SourceEntry::from_deb822_stanza)
.collect()
}
/// Parse a 'source list' string in legacy format into a SourceEntry vector
pub fn parse_legacy(data: &str) -> Vec<SourceEntry> {
data.split("\n")
data.split('\n')
.flat_map(SourceEntry::from_legacy)
.collect()
}
/// Load sources from context (or current context by default)
pub fn load(ctx: Option<Arc<crate::context::Context>>) -> Result<Vec<SourceEntry>, Box<dyn Error>> {
///
/// Reads the deb822 distro sources (ubuntu.sources or debian.sources), the
/// legacy '/etc/apt/sources.list' and the pkh-owned added-sources file when
/// they exist. Every entry remembers the file and format it came from.
pub fn load(ctx: Option<Arc<Context>>) -> Result<Vec<SourceEntry>, Box<dyn Error>> {
let mut sources = Vec::new();
let ctx = ctx.unwrap_or_else(context::current);
// Try DEB822 format first (Ubuntu 24.04+ and Debian Trixie+)
if let Ok(entries) = load_deb822(&ctx, "/etc/apt/sources.list.d/ubuntu.sources") {
sources.extend(entries);
} else if let Ok(entries) = load_deb822(&ctx, "/etc/apt/sources.list.d/debian.sources") {
sources.extend(entries);
}
load_file(
&ctx,
"/etc/apt/sources.list.d/ubuntu.sources",
SourceFormat::Deb822,
&mut sources,
)?;
load_file(
&ctx,
"/etc/apt/sources.list.d/debian.sources",
SourceFormat::Deb822,
&mut sources,
)?;
// Fall back to legacy format
if let Ok(entries) = load_legacy(&ctx, "/etc/apt/sources.list") {
sources.extend(entries);
}
load_file(
&ctx,
"/etc/apt/sources.list",
SourceFormat::Legacy,
&mut sources,
)?;
// Entries added by a previous pkh run
load_file(&ctx, PKH_ADDED_PATH, SourceFormat::Legacy, &mut sources)?;
Ok(sources)
}
/// Save sources back to context
pub fn save_legacy(
ctx: Option<Arc<crate::context::Context>>,
sources: Vec<SourceEntry>,
path: &str,
) -> Result<(), Box<dyn Error>> {
let ctx = if let Some(c) = ctx {
c
/// Save sources back to the context
///
/// Each entry is written back to the file it was loaded from
/// ([`SourceEntry::origin`]), in that file's format. Entries without an
/// origin (e.g. repositories added by pkh) go to the pkh-owned
/// added-sources file in legacy format, never to distro-managed files.
///
/// Files whose rendered content is byte-identical to their current content
/// are left untouched; otherwise a '<path>.pkh-backup' copy is created once
/// before the first overwrite.
pub fn save(ctx: Option<Arc<Context>>, sources: Vec<SourceEntry>) -> Result<(), Box<dyn Error>> {
let ctx = ctx.unwrap_or_else(context::current);
for (path, _format, content) in plan_writes(&sources) {
let original = if ctx.exists(&path)? {
Some(ctx.read_file(&path)?)
} else {
context::current()
None
};
if original.as_deref() == Some(content.as_str()) {
// Nothing changed: leave the file untouched
continue;
}
// One-time backup before overwriting an existing file
if original.is_some() {
let backup = backup_path(&path);
if !ctx.exists(&backup)? {
ctx.copy_path(&path, &backup)?;
}
}
ctx.write_file(&path, &content)?;
}
let content = sources
.into_iter()
.map(|s| s.to_legacy())
.collect::<Vec<_>>()
.join("\n");
ctx.write_file(Path::new(path), &content)?;
Ok(())
}
/// Load sources from DEB822 format
fn load_deb822(ctx: &context::Context, path: &str) -> Result<Vec<SourceEntry>, Box<dyn Error>> {
let path = Path::new(path);
if path.exists() {
let content = ctx.read_file(path)?;
return Ok(parse_deb822(&content));
/// Load entries from one sources file, if it exists, tagging them with
/// their origin
fn load_file(
ctx: &Context,
path: &str,
format: SourceFormat,
out: &mut Vec<SourceEntry>,
) -> Result<(), Box<dyn Error>> {
let path = PathBuf::from(path);
if !ctx.exists(&path)? {
return Ok(());
}
Ok(Vec::new())
let content = ctx.read_file(&path)?;
let mut entries = match format {
SourceFormat::Deb822 => parse_deb822(&content),
SourceFormat::Legacy => parse_legacy(&content),
};
for entry in &mut entries {
entry.origin = Some(SourceOrigin {
path: path.clone(),
format,
});
}
out.append(&mut entries);
Ok(())
}
/// Load sources from legacy format
fn load_legacy(ctx: &context::Context, path: &str) -> Result<Vec<SourceEntry>, Box<dyn Error>> {
let path = Path::new(path);
if path.exists() {
let content = ctx.read_file(path)?;
return Ok(content.lines().flat_map(SourceEntry::from_legacy).collect());
/// Compute the writes needed to persist entries: one
/// (path, format, content) triple per destination file, entries kept in order
///
/// Entries without an origin are routed to the pkh-owned added-sources file.
fn plan_writes(sources: &[SourceEntry]) -> Vec<(PathBuf, SourceFormat, String)> {
let mut plan: Vec<(PathBuf, SourceFormat, Vec<&SourceEntry>)> = Vec::new();
for entry in sources {
let (path, format) = match &entry.origin {
Some(origin) => (origin.path.clone(), origin.format),
None => (PathBuf::from(PKH_ADDED_PATH), SourceFormat::Legacy),
};
if let Some((_, _, group)) = plan.iter_mut().find(|(p, _, _)| *p == path) {
group.push(entry);
} else {
plan.push((path, format, vec![entry]));
}
}
Ok(Vec::new())
plan.into_iter()
.map(|(path, format, entries)| {
let content = match format {
// Legacy entries end with '\n': plain concatenation, no
// blank lines in between
SourceFormat::Legacy => entries.iter().map(|e| e.to_legacy()).collect(),
// deb822 stanzas end with '\n': a '\n' join gives one blank
// line between stanzas
SourceFormat::Deb822 => entries
.iter()
.map(|e| e.to_deb822())
.collect::<Vec<_>>()
.join("\n"),
};
(path, format, content)
})
.collect()
}
/// Backup path for a sources file ('<path>.pkh-backup')
fn backup_path(path: &Path) -> PathBuf {
let mut with_suffix = path.as_os_str().to_os_string();
with_suffix.push(BACKUP_SUFFIX);
PathBuf::from(with_suffix)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::context::ContextConfig;
#[tokio::test]
async fn test_parse_deb822() {
@@ -333,4 +558,208 @@ mod tests {
assert_eq!(sources[2].suite, vec!["resolute-security"]);
assert_eq!(sources[2].components, vec!["main"]);
}
/// Legacy round-trip: kind, signed-by, trusted and arch are preserved,
/// and rendering introduces no blank lines
#[test]
fn legacy_roundtrip_preserves_options() {
let input = "\
deb [arch=amd64 signed-by=/k.gpg] http://x noble main\n\
deb-src http://x noble main\n\
deb [trusted=yes] http://x noble universe\n\
# deb [arch=i386] http://x noble main\n";
let sources = parse_legacy(input);
assert_eq!(sources.len(), 4);
assert_eq!(sources[0].kind, SourceKind::Deb);
assert_eq!(sources[0].signed_by.as_deref(), Some("/k.gpg"));
assert_eq!(sources[0].architectures, vec!["amd64"]);
assert_eq!(sources[1].kind, SourceKind::DebSrc);
assert_eq!(sources[2].trusted, Some(true));
assert!(!sources[3].enabled);
// Render as a legacy file through the save planning path
let origin = SourceOrigin {
path: PathBuf::from("/etc/apt/sources.list"),
format: SourceFormat::Legacy,
};
let mut sources = sources;
for entry in &mut sources {
entry.origin = Some(origin.clone());
}
let plan = plan_writes(&sources);
assert_eq!(plan.len(), 1);
let rendered = &plan[0].2;
// Rendering is faithful: byte-identical and without blank lines
assert_eq!(rendered, input);
assert!(!rendered.contains("\n\n"));
let reparsed = parse_legacy(rendered);
assert_eq!(reparsed, parse_legacy(input));
}
/// deb822 round-trip: multiple types are split into one entry per type,
/// Signed-By and Enabled are preserved
#[test]
fn deb822_roundtrip_preserves_types_and_options() {
let input = "\
Types: deb deb-src\n\
URIs: http://archive.ubuntu.com/ubuntu\n\
Suites: noble\n\
Components: main\n\
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg\n\
Enabled: false\n\
\n\
Types: deb\n\
URIs: http://archive.ubuntu.com/ubuntu\n\
Suites: noble-updates\n\
Components: main universe\n";
let sources = parse_deb822(input);
// The first stanza declares two types: one entry per type
assert_eq!(sources.len(), 3);
assert_eq!(sources[0].kind, SourceKind::Deb);
assert_eq!(sources[1].kind, SourceKind::DebSrc);
assert_eq!(sources[2].kind, SourceKind::Deb);
assert!(!sources[0].enabled);
assert!(!sources[1].enabled);
assert!(sources[2].enabled);
assert_eq!(
sources[0].signed_by.as_deref(),
Some("/usr/share/keyrings/ubuntu-archive-keyring.gpg")
);
assert_eq!(sources[1].signed_by, sources[0].signed_by);
assert_eq!(sources[2].signed_by, None);
// Render as a deb822 file through the save planning path
let mut sources = sources;
for entry in &mut sources {
entry.origin = Some(SourceOrigin {
path: PathBuf::from("/etc/apt/sources.list.d/ubuntu.sources"),
format: SourceFormat::Deb822,
});
}
let plan = plan_writes(&sources);
assert_eq!(plan.len(), 1);
let rendered = &plan[0].2;
let reparsed = parse_deb822(rendered);
// Parse/render round-trip preserves the model (origin excepted)
assert_eq!(reparsed, parse_deb822(input));
assert_eq!(reparsed[0].kind, SourceKind::Deb);
assert_eq!(reparsed[1].kind, SourceKind::DebSrc);
assert_eq!(reparsed[2].kind, SourceKind::Deb);
assert!(!reparsed[0].enabled);
assert!(!reparsed[1].enabled);
assert!(reparsed[2].enabled);
assert_eq!(
reparsed[0].signed_by.as_deref(),
Some("/usr/share/keyrings/ubuntu-archive-keyring.gpg")
);
// 'Enabled' is only emitted for disabled entries
assert_eq!(rendered.matches("Enabled: no").count(), 2);
}
/// Entries are routed to their origin file in its own format, and new
/// entries (no origin) go to the pkh-owned added-sources file
#[test]
fn plan_writes_routes_by_origin() {
let origin_a = SourceOrigin {
path: PathBuf::from("/etc/apt/sources.list.d/ubuntu.sources"),
format: SourceFormat::Deb822,
};
let mut sources = parse_deb822(
"Types: deb\nURIs: http://archive.ubuntu.com/ubuntu\nSuites: noble\nComponents: main\n",
);
sources[0].origin = Some(origin_a.clone());
// Modify the origin-A entry and add a brand new (PPA) entry
sources[0].components.push("universe".to_string());
sources.push(SourceEntry {
enabled: true,
kind: SourceKind::Deb,
components: vec!["main".to_string()],
architectures: vec![],
signed_by: None,
trusted: None,
uri: "http://ppa.example.org/user/ppa/ubuntu".to_string(),
suite: vec!["noble".to_string()],
origin: None,
});
let plan = plan_writes(&sources);
assert_eq!(plan.len(), 2);
assert_eq!(plan[0].0, origin_a.path);
assert_eq!(plan[0].1, SourceFormat::Deb822);
assert!(plan[0].2.starts_with("Types: deb\n"));
assert!(plan[0].2.contains("main universe"));
assert_eq!(
plan[1].0,
PathBuf::from("/etc/apt/sources.list.d/pkh-added.list")
);
assert_eq!(plan[1].1, SourceFormat::Legacy);
assert!(plan[1].2.starts_with("deb http://ppa.example.org/"));
}
/// save() leaves unchanged files untouched, and backs up existing files
/// once before overwriting them; the backup also works for the
/// pkh-owned added-sources file
#[test]
fn save_skips_unchanged_and_backs_up() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("ubuntu.sources");
std::fs::write(
&path,
"Types: deb\nURIs: http://a\nSuites: noble\nComponents: main\n",
)
.unwrap();
let ctx = Arc::new(Context::new(ContextConfig::Local).unwrap());
let origin = SourceOrigin {
path: path.clone(),
format: SourceFormat::Deb822,
};
let mut entries =
parse_deb822("Types: deb\nURIs: http://a\nSuites: noble\nComponents: main\n");
entries[0].origin = Some(origin.clone());
// Unchanged content: no write, no backup
save(Some(ctx.clone()), entries.clone()).unwrap();
assert_eq!(
std::fs::read_to_string(&path).unwrap(),
"Types: deb\nURIs: http://a\nSuites: noble\nComponents: main\n"
);
assert!(!backup_path(&path).exists());
// Modified content: backup created, file rewritten in its own format
entries[0].components.push("universe".to_string());
save(Some(ctx.clone()), entries).unwrap();
assert_eq!(
std::fs::read_to_string(backup_path(&path)).unwrap(),
"Types: deb\nURIs: http://a\nSuites: noble\nComponents: main\n"
);
assert_eq!(
std::fs::read_to_string(&path).unwrap(),
"Types: deb\nURIs: http://a\nSuites: noble\nComponents: main universe\n"
);
// A second save does not overwrite the first backup
let mut entries =
parse_deb822("Types: deb\nURIs: http://a\nSuites: noble\nComponents: main universe\n");
entries[0].origin = Some(origin);
entries[0].components.push("restricted".to_string());
save(Some(ctx.clone()), entries).unwrap();
assert_eq!(
std::fs::read_to_string(backup_path(&path)).unwrap(),
"Types: deb\nURIs: http://a\nSuites: noble\nComponents: main\n"
);
assert_eq!(
std::fs::read_to_string(&path).unwrap(),
"Types: deb\nURIs: http://a\nSuites: noble\nComponents: main universe restricted\n"
);
}
}
+317 -137
View File
@@ -14,9 +14,7 @@ use std::path::{Path, PathBuf};
use std::sync::Arc;
use crate::context::Context;
use crate::debian::{
ChecksumEntry, ControlInfo, FileChecksums, FilesList, parse_changelog_entry_from_str,
};
use crate::debian::{ChecksumEntry, ControlInfo, FileChecksums, FilesList};
/// Digests of one artifact.
#[derive(Debug, Clone, Default)]
@@ -34,10 +32,13 @@ pub struct BinaryMetadataOptions {
pub profiles: Vec<String>,
/// Vendor name (`Build-Origin`).
pub vendor: String,
/// Parallel job count advertised in `DEB_BUILD_OPTIONS`.
pub parallel: usize,
/// Reproducible-builds epoch exported to the build.
pub source_date_epoch: i64,
/// Environment variables pkh exported to the build steps (e.g. `LANG`,
/// `DEB_BUILD_OPTIONS` with the real parallel count and `nocheck`,
/// `SOURCE_DATE_EPOCH`, cross `DEB_*` variables). Recorded — filtered to
/// dpkg's allow-list — in the `.buildinfo` `Environment` field, taking
/// precedence over whatever the host process inherited, so the metadata
/// describes the environment the build actually ran in.
pub exported_env: BTreeMap<String, String>,
/// Build architecture (the machine inside the build context).
pub build_arch: String,
/// Host architecture (the packages' target); equals the build
@@ -54,7 +55,7 @@ pub struct BinaryMetadataOptions {
/// `dpkg-genchanges -b`: sorted `Binary` list, encounter-order `Architecture`
/// accumulation, sorted `Description` lines formatted like dpkg, `.buildinfo`
/// registration in `debian/files`, and binary-NMU handling (`Source:
/// pkg (prev)` + previous `.dsc` redistribution when present).
/// pkg (prev)` + `Binary-Only-Changes`, with no source files distributed).
pub fn generate_binary_metadata(
ctx: &Arc<Context>,
package_dir: &Path,
@@ -65,14 +66,25 @@ pub fn generate_binary_metadata(
// Metadata sources inside the context
// ------------------------------------------------------------------
let changelog_content = ctx.read_file(&package_dir.join("debian/changelog"))?;
let entry = parse_changelog_entry_from_str(&changelog_content)?;
let mut entries =
crate::debian::changelog::parse_changelog_entries_from_str(&changelog_content, Some(2))?;
let entry = entries.remove(0);
let previous_entry = entries.into_iter().next();
let control_content = ctx.read_file(&package_dir.join("debian/control"))?;
let control = ControlInfo::parse_content(&control_content)?;
let files_content = ctx
.read_file(&package_dir.join("debian/files"))
.unwrap_or_default();
// A missing `debian/files` is tolerated (first binary build in a fresh
// tree has nothing registered yet; that surfaces below as the "no binary
// artifacts" error), like `FilesList::load`. Any other read failure must
// not be silently mistaken for an empty registry.
let files_path = package_dir.join("debian/files");
let files_content = if ctx.exists(&files_path)? {
ctx.read_file(&files_path)
.map_err(|e| format!("cannot read '{}': {}", files_path.display(), e))?
} else {
String::new()
};
let mut files_list = FilesList::parse(&files_content)?;
// ------------------------------------------------------------------
@@ -97,6 +109,9 @@ pub fn generate_binary_metadata(
let entry_hashes = hashes
.remove(name)
.ok_or_else(|| format!("artifact '{name}' listed in debian/files but not found"))?;
// SHA-512 stays unknown here: like dpkg-genbuildinfo, no SHA-512
// digest is computed for the artifacts, and an empty digest keeps
// the `Checksums-Sha512` field of the `.buildinfo` omitted.
checksums.insert_entry(
name,
ChecksumEntry {
@@ -104,6 +119,7 @@ pub fn generate_binary_metadata(
md5: entry_hashes.md5,
sha1: entry_hashes.sha1,
sha256: entry_hashes.sha256,
sha512: String::new(),
},
);
// Architecture accumulation in encounter order (dpkg-genchanges).
@@ -119,28 +135,24 @@ pub fn generate_binary_metadata(
}
// ------------------------------------------------------------------
// Binary-NMU: redistribute the previous source when present
// Binary-NMU: reference the previous source version, textually only
// ------------------------------------------------------------------
let sversion = entry.version.no_epoch();
let mut source_display = entry.source.clone();
let mut binary_only_changes = None;
if entry.binary_only
&& let Ok(prev_entry) = crate::debian::changelog::parse_previous_version_from_str(
&ctx.read_file(&package_dir.join("debian/changelog"))?,
)
&& let Some(prev) = prev_entry
{
source_display = format!("{} ({})", entry.source, prev);
if entry.binary_only {
// Like dpkg-genchanges/genbuildinfo, a binary-only upload references
// the previous source version in the `Source` field and records the
// entry in `Binary-Only-Changes`, but distributes NO source files:
// the previous `.dsc` and its tarballs already sit in the archive,
// and are not re-uploaded even when present next to the tree.
if let Some(prev) = &previous_entry {
source_display = format!("{} ({})", entry.source, prev.version.full());
binary_only_changes = Some(format!(
"{}\n\n -- {} <{}> {}",
entry.changes_field, entry.maintainer_name, entry.maintainer_email, entry.date_raw
));
let prev_version = crate::debian::DebianVersion::parse(&prev)?;
let dsc_name = format!("{}_{}.dsc", entry.source, prev_version.no_epoch());
let dsc_path = upload_dir.join(&dsc_name);
if ctx.exists(&dsc_path)? {
include_dsc_artifacts(ctx, upload_dir, &dsc_name, &mut checksums)?;
}
}
@@ -194,9 +206,13 @@ pub fn generate_binary_metadata(
// ------------------------------------------------------------------
// Installed-Build-Depends closure over the context status database
// ------------------------------------------------------------------
// Like the source-build path, a status database that cannot be read is
// a hard error: silently treating it as empty would drop (or gut) the
// `Installed-Build-Depends` field of the produced metadata.
let status_path = Path::new("/var/lib/dpkg/status");
let status_content = ctx
.read_file(Path::new("/var/lib/dpkg/status"))
.unwrap_or_default();
.read_file(status_path)
.map_err(|e| format!("cannot read status file '{}': {}", status_path.display(), e))?;
let bd_fields = [
control.source.get("Build-Depends").unwrap_or(""),
control.source.get("Build-Depends-Arch").unwrap_or(""),
@@ -208,8 +224,10 @@ pub fn generate_binary_metadata(
// ------------------------------------------------------------------
// .buildinfo generation, then registration in debian/files
// ------------------------------------------------------------------
let pipeline_env = pipeline_environment(opts);
let environment = crate::build::env::buildinfo_environment(&pipeline_env);
// Record exactly the environment that was exported to the build steps,
// overriding any host-inherited value (dpkg-style allowed-variable
// filtering, export precedence).
let environment = crate::build::env::buildinfo_environment(&opts.exported_env);
// dpkg-genbuildinfo sorts the accumulated architecture values, while
// dpkg-genchanges keeps encounter order.
@@ -257,6 +275,9 @@ pub fn generate_binary_metadata(
md5: h.md5.clone(),
sha1: h.sha1.clone(),
sha256: h.sha256.clone(),
// No SHA-512 digest available (see above); keeps the
// `Checksums-Sha512` `.buildinfo` field omitted.
sha512: String::new(),
},
);
}
@@ -270,6 +291,7 @@ pub fn generate_binary_metadata(
date: entry.date_raw.clone(),
source: source_display,
binaries,
binary_only: entry.binary_only,
built_for_profiles: opts.profiles.clone(),
architecture: arch_values.join(" "),
version: entry.version.full(),
@@ -292,24 +314,6 @@ pub fn generate_binary_metadata(
Ok((buildinfo_path, changes_path))
}
/// Environment exported to the build steps; recorded (filtered) in the
/// `.buildinfo` `Environment` field.
fn pipeline_environment(opts: &BinaryMetadataOptions) -> BTreeMap<String, String> {
let mut env = BTreeMap::new();
env.insert(
"SOURCE_DATE_EPOCH".to_string(),
opts.source_date_epoch.to_string(),
);
env.insert(
"DEB_BUILD_OPTIONS".to_string(),
format!("parallel={}", opts.parallel),
);
if !opts.profiles.is_empty() {
env.insert("DEB_BUILD_PROFILES".to_string(), opts.profiles.join(","));
}
env
}
/// Compute md5/sha1/sha256 digests and sizes for the named files inside the
/// context directory `dir`, using coreutils.
fn hashes_in_context(
@@ -322,21 +326,34 @@ fn hashes_in_context(
.map(|n| (n.clone(), ArtifactHashes::default()))
.collect();
// Sizes.
// Sizes. A failed `stat` must fail the metadata generation: an unchecked
// exit status would leave the default size 0 in the produced
// `.changes`/`.buildinfo` checksum entries.
let output = ctx
.command("stat")
.current_dir(dir)
.arg("-c")
.arg("%s %n")
.args(names)
.output()?;
.output()
.map_err(|e| format!("failed to run 'stat' inside the build context: {e}"))?;
if !output.status.success() {
return Err(format!(
"'stat' failed inside the build context: {}",
String::from_utf8_lossy(&output.stderr).trim()
)
.into());
}
let stdout = String::from_utf8_lossy(&output.stdout);
for line in stdout.lines() {
let Some((size, name)) = line.trim().split_once(' ') else {
continue;
};
let size = size
.parse::<u64>()
.map_err(|_| format!("'stat' reported an invalid size '{size}' for '{name}'"))?;
if let Some(slot) = out.get_mut(name) {
slot.size = size.parse().unwrap_or(0);
slot.size = size;
}
}
@@ -378,98 +395,261 @@ fn hashes_in_context(
Ok(out)
}
/// Pull the `.dsc` checksums (and its referenced tarballs) into the
/// checksum registry, mirroring how binary-NMU uploads redistribute the
/// previous source.
fn include_dsc_artifacts(
ctx: &Arc<Context>,
upload_dir: &Path,
dsc_name: &str,
checksums: &mut FileChecksums,
) -> Result<(), Box<dyn Error>> {
let dsc_content = ctx.read_file(&upload_dir.join(dsc_name))?;
let para = crate::debian::control::parse_paragraphs(&dsc_content)
.into_iter()
.next()
.ok_or_else(|| format!("'{dsc_name}' is empty"))?;
#[cfg(test)]
mod tests {
use super::*;
let mut names: Vec<String> = Vec::new();
let mut partials: BTreeMap<String, PartialDscChecksums> = BTreeMap::new();
for field in ["Checksums-Sha1", "Checksums-Sha256"] {
if let Some(value) = para.get(field) {
for line in value.lines() {
let tokens: Vec<&str> = line.split_whitespace().collect();
if tokens.len() != 3 {
continue;
}
let slot = partials.entry(tokens[2].to_string()).or_default();
if field == "Checksums-Sha1" {
slot.sha1 = Some(tokens[0].to_string());
} else {
slot.sha256 = Some(tokens[0].to_string());
}
slot.size = tokens[1].parse().ok().or(slot.size);
}
}
}
if let Some(files_value) = para.get("Files") {
for line in files_value.lines() {
let tokens: Vec<&str> = line.split_whitespace().collect();
if tokens.len() >= 3 {
let slot = partials.entry(tokens[2].to_string()).or_default();
slot.md5 = Some(tokens[0].to_string());
slot.size = tokens[1].parse().ok().or(slot.size);
}
}
}
for field in ["Checksums-Sha1", "Checksums-Sha256"] {
if let Some(value) = para.get(field) {
for line in value.lines() {
if let Some(name) = line.split_whitespace().nth(2) {
names.push(name.to_string());
}
}
}
/// The recorded `.buildinfo` `Environment` must carry the environment
/// actually exported to the build steps (`parallel=N nocheck`, `LANG=C`,
/// ...), taking precedence over any host-inherited value, instead of
/// values recomputed from host state at generation time.
#[test]
fn environment_records_exported_env_not_host_defaults() {
let mut exported_env = BTreeMap::new();
exported_env.insert("LANG".to_string(), "C".to_string());
exported_env.insert(
"DEB_BUILD_OPTIONS".to_string(),
"parallel=7 nocheck".to_string(),
);
let opts = BinaryMetadataOptions {
profiles: Vec::new(),
vendor: "debian".to_string(),
exported_env,
build_arch: "amd64".to_string(),
host_arch: "amd64".to_string(),
};
let environment = crate::build::env::buildinfo_environment(&opts.exported_env);
assert!(
environment.contains("DEB_BUILD_OPTIONS=\"parallel=7 nocheck\""),
"recorded Environment must carry the exported DEB_BUILD_OPTIONS: {environment}"
);
assert!(
environment.contains("LANG=\"C\""),
"recorded Environment must carry the exported LANG: {environment}"
);
// Not in dpkg's allowed-variable list: never recorded.
assert!(!environment.contains("DEBIAN_FRONTEND"), "{environment}");
}
// The .dsc itself is hashed fresh (it may be signed/rewritten); the
// tarballs reuse the .dsc-recorded digests, like dpkg-genchanges does.
let dsc_hashes =
hashes_in_context(ctx, upload_dir, std::slice::from_ref(&dsc_name.to_string()))?;
if let Some(h) = dsc_hashes.get(dsc_name) {
checksums.insert_entry(
dsc_name,
ChecksumEntry {
size: h.size,
md5: h.md5.clone(),
sha1: h.sha1.clone(),
sha256: h.sha256.clone(),
},
/// A binary-only (binNMU) build whose changelog cannot yield the
/// previous entry (malformed second header, unbalanced parenthesis) must
/// fail the metadata generation with a diagnostic naming the problem,
/// instead of silently emitting a plain `Source:` `.changes` with no
/// `Binary-Only-Changes` and no previous-version reference.
#[test]
fn binary_only_prev_version_parse_failure_errors_instead_of_wrong_metadata() {
let changelog = "\
hello (1.0-1+b1) unstable; urgency=medium, binary-only=yes
* Binary-only rebuild.
-- A B <a@b.c> Mon, 01 Jan 2024 00:00:00 +0000
hello (1.0-1 unstable; urgency=medium
* Previous entry with an unbalanced parenthesis.
-- A B <a@b.c> Sun, 31 Dec 2023 00:00:00 +0000
";
let control = "\
Source: hello
Section: devel
Priority: optional
Maintainer: A B <a@b.c>
Package: hello
Architecture: all
Description: test package
";
let base = tempfile::tempdir().expect("tempdir");
let tree = base.path().join("hello-1.0");
std::fs::create_dir_all(tree.join("debian")).expect("mkdir tree");
std::fs::write(tree.join("debian/changelog"), changelog).expect("write changelog");
std::fs::write(tree.join("debian/control"), control).expect("write control");
std::fs::write(
tree.join("debian/files"),
"hello_1.0-1+b1_all.deb devel optional\n",
)
.expect("write files");
std::fs::write(base.path().join("hello_1.0-1+b1_all.deb"), "deb payload")
.expect("write deb");
let ctx = Arc::new(
crate::context::Context::new(crate::context::ContextConfig::Local).expect("context"),
);
let opts = BinaryMetadataOptions {
profiles: Vec::new(),
vendor: "debian".to_string(),
exported_env: BTreeMap::new(),
build_arch: "amd64".to_string(),
host_arch: "amd64".to_string(),
};
let err = generate_binary_metadata(&ctx, &tree, base.path(), &opts)
.expect_err("binary-only build with an unparseable changelog must fail");
let err = err.to_string();
assert!(err.contains("unbalanced parenthesis"), "{err}");
assert!(err.contains("1.0-1 unstable"), "{err}");
}
/// An unreadable `debian/files` (e.g. permissions) must fail the
/// metadata generation with an error naming the read failure, instead of
/// being silently treated as an empty registry and reported as "no
/// binary artifacts found". A *missing* file stays tolerated (first
/// build in a fresh tree); the distinction matters.
#[test]
fn unreadable_debian_files_errors_instead_of_empty_registry() {
if crate::utils::root::is_root().unwrap_or(false) {
// Root can read files regardless of permissions.
return;
}
let changelog = "\
hello (1.0-1) unstable; urgency=medium
* Regular build.
-- A B <a@b.c> Mon, 01 Jan 2024 00:00:00 +0000
";
let control = "\
Source: hello
Section: devel
Priority: optional
Maintainer: A B <a@b.c>
Package: hello
Architecture: all
Description: test package
";
let base = tempfile::tempdir().expect("tempdir");
let tree = base.path().join("hello-1.0");
std::fs::create_dir_all(tree.join("debian")).expect("mkdir tree");
std::fs::write(tree.join("debian/changelog"), changelog).expect("write changelog");
std::fs::write(tree.join("debian/control"), control).expect("write control");
let files_path = tree.join("debian/files");
std::fs::write(&files_path, "hello_1.0-1_all.deb devel optional\n").expect("write files");
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&files_path, std::fs::Permissions::from_mode(0o000))
.expect("chmod files");
}
let ctx = Arc::new(
crate::context::Context::new(crate::context::ContextConfig::Local).expect("context"),
);
let opts = BinaryMetadataOptions {
profiles: Vec::new(),
vendor: "debian".to_string(),
exported_env: BTreeMap::new(),
build_arch: "amd64".to_string(),
host_arch: "amd64".to_string(),
};
let err = generate_binary_metadata(&ctx, &tree, base.path(), &opts)
.expect_err("unreadable debian/files must fail with a read error");
let err = err.to_string();
assert!(err.contains("cannot read"), "{err}");
assert!(err.contains("debian/files"), "{err}");
#[cfg(unix)]
assert!(err.contains("Permission denied"), "{err}");
}
/// A binary-only (binNMU) build references the previous source version
/// (`Source: pkg (prev)`, `Binary-Only-Changes`) but must NOT
/// redistribute any source file: like dpkg-genchanges/genbuildinfo, the
/// previous `.dsc` and its tarballs stay out of both documents even when
/// they exist next to the artifacts.
#[test]
fn binary_only_metadata_references_previous_source_without_redistributing_it() {
let changelog = "\
hello (1.0-1+b1) unstable; urgency=medium, binary-only=yes
* Binary-only rebuild.
-- A B <a@b.c> Mon, 01 Jan 2024 00:00:00 +0000
hello (1.0-1) unstable; urgency=medium
* Initial release.
-- A B <a@b.c> Sun, 31 Dec 2023 00:00:00 +0000
";
let control = "\
Source: hello
Section: devel
Priority: optional
Maintainer: A B <a@b.c>
Package: hello
Architecture: all
Description: test package
";
let base = tempfile::tempdir().expect("tempdir");
let tree = base.path().join("hello-1.0");
std::fs::create_dir_all(tree.join("debian")).expect("mkdir tree");
std::fs::write(tree.join("debian/changelog"), changelog).expect("write changelog");
std::fs::write(tree.join("debian/control"), control).expect("write control");
std::fs::write(
tree.join("debian/files"),
"hello_1.0-1+b1_all.deb devel optional\n",
)
.expect("write files");
std::fs::write(base.path().join("hello_1.0-1+b1_all.deb"), "deb payload")
.expect("write deb");
// The trap: the previous source artifacts sit right next to the
// binaries, as they would after a source build. dpkg does not
// redistribute them for a binary-only upload, and neither must we.
std::fs::write(
base.path().join("hello_1.0-1.dsc"),
"Format: 3.0 (quilt)\nSource: hello\nBinary: hello\nArchitecture: any\nVersion: \
1.0-1\nMaintainer: A B <a@b.c>\nChecksums-Sha1:\n aaa111 12 \
hello_1.0.orig.tar.xz\n",
)
.expect("write previous dsc");
std::fs::write(base.path().join("hello_1.0.orig.tar.xz"), "tarball bytes")
.expect("write previous tarball");
let ctx = Arc::new(
crate::context::Context::new(crate::context::ContextConfig::Local).expect("context"),
);
let opts = BinaryMetadataOptions {
profiles: Vec::new(),
vendor: "debian".to_string(),
exported_env: BTreeMap::new(),
build_arch: "amd64".to_string(),
host_arch: "amd64".to_string(),
};
let (buildinfo_path, changes_path) =
generate_binary_metadata(&ctx, &tree, base.path(), &opts)
.expect("binNMU metadata generation must succeed");
let changes = std::fs::read_to_string(&changes_path).expect("read changes");
let buildinfo = std::fs::read_to_string(&buildinfo_path).expect("read buildinfo");
// The previous version is referenced textually.
assert!(
changes.contains("Source: hello (1.0-1)"),
"changes must reference the previous version: {changes}"
);
assert!(
buildinfo.contains("Binary-Only-Changes"),
"buildinfo must record the binary-only entry: {buildinfo}"
);
// ... but no source file is distributed, on either side.
for (doc, text) in [("changes", &changes), ("buildinfo", &buildinfo)] {
assert!(
!text.contains("hello_1.0-1.dsc"),
"{doc} must not redistribute the previous .dsc: {text}"
);
assert!(
!text.contains("hello_1.0.orig.tar.xz"),
"{doc} must not redistribute the previous tarball: {text}"
);
}
for name in &names {
if name == dsc_name {
continue;
}
let p = &partials[name];
checksums.insert_entry(
name,
ChecksumEntry {
size: p.size.unwrap_or(0),
md5: p.md5.clone().unwrap_or_default(),
sha1: p.sha1.clone().unwrap_or_default(),
sha256: p.sha256.clone().unwrap_or_default(),
},
// The distributed set is exactly the binary artifacts + buildinfo.
assert!(
changes.contains("hello_1.0-1+b1_all.deb") && changes.contains(".buildinfo"),
"changes must distribute the deb and the buildinfo: {changes}"
);
}
Ok(())
}
/// Partially-known checksums taken from a `.dsc` checksum field.
#[derive(Debug, Default)]
struct PartialDscChecksums {
size: Option<u64>,
md5: Option<String>,
sha1: Option<String>,
sha256: Option<String>,
}
+149
View File
@@ -207,6 +207,13 @@ pub fn installed_build_depends_from_content(
entries.sort_by(|a, b| a.0.cmp(&b.0).then_with(|| a.1.cmp(&b.1)));
entries.dedup_by(|a, b| a.0 == b.0);
// With no reachable entries, return an empty value so `render_buildinfo`
// omits the field entirely; a leading `\n` alone would render a
// malformed `Installed-Build-Depends:` with only a blank continuation.
if entries.is_empty() {
return Ok(String::new());
}
// Leading `\n`: pre-wrapped multiline field, dpkg-style.
let mut out = String::from("\n");
out.push_str(
@@ -289,6 +296,12 @@ pub fn render_buildinfo(input: &BuildInfoInput) -> Paragraph {
p.set("Checksums-Md5", &input.checksums.field_md5());
p.set("Checksums-Sha1", &input.checksums.field_sha1());
p.set("Checksums-Sha256", &input.checksums.field_sha256());
// Only-if-populated: entries merged from a `.dsc` carry no SHA-512
// (dpkg only records sha1/sha256 there), and an incomplete checksum
// list must never be rendered.
if let Some(sha512) = input.checksums.field_sha512() {
p.set("Checksums-Sha512", &sha512);
}
}
p.set("Build-Origin", &input.build_origin);
p.set("Build-Architecture", &input.build_architecture);
@@ -378,6 +391,68 @@ Architecture: amd64
assert!(!ibd.contains("not-installed"));
}
/// With no installed entries reachable (empty status database), the
/// computed value must be EMPTY so `render_buildinfo` omits the
/// `Installed-Build-Depends` field entirely, instead of emitting a
/// malformed field with only a blank continuation line.
#[test]
fn installed_build_depends_without_entries_is_empty_and_omitted() {
let ibd = installed_build_depends_from_content("", &["libc6"]).unwrap();
assert_eq!(
ibd, "",
"zero entries must yield an empty value, not \"\\n\""
);
let input = BuildInfoInput {
source: "hello".to_string(),
binaries: vec!["hello".to_string()],
architecture: "amd64".to_string(),
version: "1.0".to_string(),
binary_only_changes: None,
build_origin: "debian".to_string(),
build_architecture: "amd64".to_string(),
build_date: "Sat, 22 Aug 2026 23:08:42 +0200".to_string(),
checksums: FileChecksums::new(),
installed_build_depends: ibd,
environment: String::new(),
};
let p = render_buildinfo(&input);
assert!(
p.get("Installed-Build-Depends").is_none(),
"empty value must omit the field entirely"
);
}
/// With installed entries, the value keeps the dpkg-style leading `\n`
/// (pre-wrapped multiline field) and the field is rendered.
#[test]
fn installed_build_depends_with_entries_renders_field() {
let status = "\
Package: gcc
Status: install ok installed
Version: 13.2
Architecture: amd64
";
let ibd = installed_build_depends_from_content(status, &["gcc"]).unwrap();
assert_eq!(ibd, "\ngcc (= 13.2)");
let input = BuildInfoInput {
source: "hello".to_string(),
binaries: vec!["hello".to_string()],
architecture: "amd64".to_string(),
version: "1.0".to_string(),
binary_only_changes: None,
build_origin: "debian".to_string(),
build_architecture: "amd64".to_string(),
build_date: "Sat, 22 Aug 2026 23:08:42 +0200".to_string(),
checksums: FileChecksums::new(),
installed_build_depends: ibd,
environment: String::new(),
};
let p = render_buildinfo(&input);
assert_eq!(p.get("Installed-Build-Depends"), Some("\ngcc (= 13.2)"));
}
#[test]
fn wrap_binary_field() {
assert_eq!(wrap_long("abc"), "abc");
@@ -425,4 +500,78 @@ Architecture: amd64
);
assert_eq!(p.get("Format"), Some("1.0"));
}
/// `Checksums-Sha512` is emitted (after `Checksums-Sha256`) only when
/// every distributed file has a SHA-512 digest; entries merged without
/// one (e.g. taken from a `.dsc`) omit the field entirely instead of
/// rendering an incomplete checksum list.
#[test]
fn checksums_sha512_emitted_only_when_populated() {
let dir = tempfile::tempdir().unwrap();
let artifact = dir.path().join("hello_1.0_all.deb");
std::fs::write(&artifact, b"deb payload").unwrap();
let mut checksums = FileChecksums::new();
checksums.add_file(&artifact).unwrap();
let mk_input = |checksums: FileChecksums| BuildInfoInput {
source: "hello".to_string(),
binaries: vec![],
architecture: "all".to_string(),
version: "1.0".to_string(),
binary_only_changes: None,
build_origin: "debian".to_string(),
build_architecture: "amd64".to_string(),
build_date: "Sat, 22 Aug 2026 23:08:42 +0200".to_string(),
checksums,
installed_build_depends: String::new(),
environment: String::new(),
};
// All digests computed: the field is present and parses back.
let p = render_buildinfo(&mk_input(checksums.clone()));
let keys: Vec<&str> = p.iter().map(|(k, _)| k).collect();
assert_eq!(
keys,
vec![
"Format",
"Source",
"Architecture",
"Version",
"Checksums-Md5",
"Checksums-Sha1",
"Checksums-Sha256",
"Checksums-Sha512",
"Build-Origin",
"Build-Architecture",
"Build-Date",
]
);
let sha512_field = p.get("Checksums-Sha512").unwrap();
let parsed =
FileChecksums::parse_field(crate::debian::ChecksumKind::Sha512, sha512_field).unwrap();
assert_eq!(parsed.len(), 1);
assert_eq!(parsed[0].0, "hello_1.0_all.deb");
assert_eq!(
parsed[0].1.sha512,
checksums.get("hello_1.0_all.deb").unwrap().sha512
);
// An entry without SHA-512 (as merged from a `.dsc`) suppresses the
// field; the other Checksums fields keep listing every file.
checksums.insert_entry(
"hello_1.0.orig.tar.xz",
crate::debian::ChecksumEntry {
size: 3,
md5: checksums.get("hello_1.0_all.deb").unwrap().md5.clone(),
sha1: String::new(),
sha256: String::new(),
sha512: String::new(),
},
);
let p = render_buildinfo(&mk_input(checksums));
assert!(p.get("Checksums-Sha512").is_none());
let sha256_lines = p.get("Checksums-Sha256").unwrap().lines();
assert_eq!(sha256_lines.filter(|l| !l.is_empty()).count(), 2);
}
}
+140
View File
@@ -4,10 +4,58 @@
use std::path::Path;
use super::OrigSourceMode;
use crate::debian::changelog::ChangelogEntry;
use crate::debian::checksums::FileChecksums;
use crate::debian::control::{Paragraph, write_paragraph};
use crate::debian::files::FilesList;
/// Compression suffixes dpkg recognizes on source tarballs.
const TARBALL_COMPRESSIONS: &[&str] = &[".gz", ".bz2", ".xz", ".lzma", ".zst"];
/// Whether this `.dsc`-listed file is an upstream orig tarball
/// (`*.orig.tar.<ext>` or a component tarball `*.orig-<c>.tar.<ext>`),
/// mirroring dpkg-genchanges' strip pattern `\.orig(-.+)?\.tar\.$ext`.
pub fn is_orig_tarball(name: &str) -> bool {
TARBALL_COMPRESSIONS.iter().any(|ext| {
name.strip_suffix(ext)
.and_then(|s| s.strip_suffix(".tar"))
.is_some_and(|stem| stem.ends_with(".orig") || stem.contains(".orig-"))
})
}
/// Whether this `.dsc`-listed file is the Debian part of the source package
/// (`*.debian.tar.<ext>` for the 3.0 formats, `*.diff.<ext>` for 1.0).
pub fn is_debian_tarball_or_diff(name: &str) -> bool {
TARBALL_COMPRESSIONS.iter().any(|ext| {
name.ends_with(&format!(".debian.tar{ext}")) || name.ends_with(&format!(".diff{ext}"))
})
}
/// Whether the upload redistributes the upstream tarballs, mirroring the
/// dpkg-genchanges source styles: `Always`/`Never` are the forced
/// `-sa`/`-sd`, while `Auto` is the default `-si` — include them only when
/// there is no previous changelog entry (first upload) or the source name or
/// upstream version changed since it. Like dpkg, the comparison uses the
/// epoch-less upstream version: a plain revision bump reuses the tarball
/// already in the archive.
pub fn include_orig_tarball(
mode: OrigSourceMode,
current: &ChangelogEntry,
previous: Option<&ChangelogEntry>,
) -> bool {
match mode {
OrigSourceMode::Always => true,
OrigSourceMode::Never => false,
OrigSourceMode::Auto => match previous {
None => true,
Some(prev) => {
prev.source != current.source || prev.version.upstream != current.version.upstream
}
},
}
}
/// Everything needed to render a `.changes` file.
#[derive(Debug, Clone)]
pub struct ChangesInput {
@@ -17,6 +65,9 @@ pub struct ChangesInput {
pub source: String,
/// Sorted binary package names with artifacts (empty for source-only).
pub binaries: Vec<String>,
/// Whether the changelog entry is a binary-only (binNMU) upload
/// (`Binary-Only: yes` field).
pub binary_only: bool,
/// Active build profiles (`Built-For-Profiles`); omitted when empty.
pub built_for_profiles: Vec<String>,
/// `Architecture` field value in encounter order (e.g. `source`,
@@ -94,6 +145,9 @@ pub fn render_changes(input: &ChangesInput) -> Paragraph {
let joined = input.binaries.join(" ");
p.set("Binary", &wrap_long(&joined));
}
if input.binary_only {
p.set("Binary-Only", "yes");
}
if !input.built_for_profiles.is_empty() {
p.set("Built-For-Profiles", &input.built_for_profiles.join(" "));
}
@@ -159,6 +213,91 @@ pub fn save_changes(path: &Path, paragraph: &Paragraph) -> Result<(), Box<dyn st
mod tests {
use super::*;
#[test]
fn orig_tarball_detection() {
assert!(is_orig_tarball("pkg_1.0.orig.tar.gz"));
assert!(is_orig_tarball("pkg_1.0.orig.tar.xz"));
assert!(is_orig_tarball("pkg_1.0.orig.tar.zst"));
assert!(is_orig_tarball("pkg_1.0~rc1.orig.tar.bz2"));
// Component tarballs.
assert!(is_orig_tarball("pkg_1.0.orig-docs.tar.xz"));
assert!(is_orig_tarball("pkg_1.0.orig-vendor.tar.gz"));
// Not orig tarballs.
assert!(!is_orig_tarball("pkg_1.0.debian.tar.xz"));
assert!(!is_debian_tarball_or_diff("pkg_1.0.orig.tar.xz"));
assert!(!is_orig_tarball("pkg_1.0.tar.xz")); // native tarball
assert!(!is_orig_tarball("pkg_1.0.dsc"));
assert!(!is_orig_tarball("pkg_1.0.orig.tar")); // no compression suffix
}
#[test]
fn debian_tarball_detection() {
assert!(is_debian_tarball_or_diff("pkg_1.0.debian.tar.xz"));
assert!(is_debian_tarball_or_diff("pkg_1.0.diff.gz"));
assert!(!is_debian_tarball_or_diff("pkg_1.0.orig.tar.xz"));
assert!(!is_debian_tarball_or_diff("pkg_1.0.tar.xz"));
}
/// Build a minimal changelog entry for one source/version pair.
fn entry(src: &str, ver: &str) -> ChangelogEntry {
crate::debian::changelog::parse_changelog_entries_from_str(
&format!(
"{src} ({ver}) unstable; urgency=medium\n\n * x\n\n \
-- A B <a@b.c> Thu, 01 Jan 2026 00:00:00 +0000\n"
),
Some(1),
)
.unwrap()
.remove(0)
}
#[test]
fn orig_inclusion_matrix() {
let cur = entry("pkg", "1.4-2");
let prev_same_upstream = entry("pkg", "1.4-1");
let prev_new_upstream = entry("pkg", "2.0-1");
let prev_renamed = entry("renamed", "1.4-1");
// -sa / -sd force the outcome.
assert!(include_orig_tarball(
OrigSourceMode::Always,
&cur,
Some(&prev_same_upstream)
));
assert!(!include_orig_tarball(
OrigSourceMode::Never,
&cur,
Some(&prev_new_upstream)
));
// -si: first upload includes; a revision bump excludes; a new
// upstream version or a renamed source includes.
assert!(include_orig_tarball(OrigSourceMode::Auto, &cur, None));
assert!(!include_orig_tarball(
OrigSourceMode::Auto,
&cur,
Some(&prev_same_upstream)
));
assert!(include_orig_tarball(
OrigSourceMode::Auto,
&cur,
Some(&prev_new_upstream)
));
assert!(include_orig_tarball(
OrigSourceMode::Auto,
&cur,
Some(&prev_renamed)
));
// The epoch is not part of the comparison, like dpkg's version().
let cur_epoch = entry("pkg", "2:1.4-2");
assert!(!include_orig_tarball(
OrigSourceMode::Auto,
&cur_epoch,
Some(&prev_same_upstream)
));
}
#[test]
fn description_formatting() {
assert_eq!(
@@ -196,6 +335,7 @@ mod tests {
date: "Sat, 22 Aug 2026 10:00:00 +0000".to_string(),
source: "pkg".to_string(),
binaries: vec![],
binary_only: false,
built_for_profiles: vec![],
architecture: "source".to_string(),
version: "1.0".to_string(),
+162 -17
View File
@@ -3,7 +3,7 @@
//! sanitized environment recorded in `.buildinfo` files.
use std::collections::BTreeMap;
use std::path::Path;
use std::path::{Path, PathBuf};
/// Number of parallel jobs to advertise in `DEB_BUILD_OPTIONS`.
pub fn num_parallel() -> usize {
@@ -12,12 +12,36 @@ pub fn num_parallel() -> usize {
.unwrap_or(1)
}
/// Merge an inherited `DEB_BUILD_OPTIONS` value with options pkh computes
/// itself.
///
/// `dpkg-buildpackage` prepends the environment's `DEB_BUILD_OPTIONS` to the
/// options it derives (`parallel=N`, ...), so caller-set options such as
/// `terse` or `nocheck` survive alongside pkh's own. The result is therefore
/// the inherited options followed by `computed`, space-separated; each side is
/// trimmed and its internal whitespace runs collapsed. An unset or blank
/// inherited value yields just `computed`.
pub fn merge_deb_build_options(inherited: Option<&str>, computed: &str) -> String {
let computed = normalize_build_options(computed);
match inherited.map(normalize_build_options) {
Some(inherited) if !inherited.is_empty() => format!("{} {}", inherited, computed),
_ => computed,
}
}
/// Trim and collapse internal whitespace in a `DEB_BUILD_OPTIONS` fragment.
fn normalize_build_options(options: &str) -> String {
options.split_whitespace().collect::<Vec<_>>().join(" ")
}
/// Compute the environment variables exported before running any build step.
///
/// Mirrors dpkg behavior:
/// - `SOURCE_DATE_EPOCH` from the changelog entry timestamp
/// (<https://reproducible-builds.org/specs/source-date-epoch/>),
/// - `DEB_BUILD_OPTIONS=parallel=N` (auto-detected job count),
/// - `DEB_BUILD_OPTIONS`: any value inherited from the invoking environment
/// (dpkg-buildpackage prepends it) followed by `parallel=N` (auto-detected
/// job count),
/// - `DEB_BUILD_PROFILES` when non-default profiles are requested.
///
/// The locale is pinned to `C` (`LC_ALL`, which takes precedence over any
@@ -38,7 +62,10 @@ pub fn build_env(
);
env.insert(
"DEB_BUILD_OPTIONS".to_string(),
format!("parallel={}", parallel),
merge_deb_build_options(
std::env::var("DEB_BUILD_OPTIONS").ok().as_deref(),
&format!("parallel={}", parallel),
),
);
if !build_profiles.is_empty() {
env.insert("DEB_BUILD_PROFILES".to_string(), build_profiles.join(","));
@@ -57,14 +84,35 @@ pub fn arch_env(host_arch: Option<&str>) -> Result<BTreeMap<String, String>, Str
crate::debian::arch::arch_env(host_arch)
}
/// Read the current vendor name from `/etc/dpkg/origins/default`
/// (its `Vendor:` or `Origin:` field), defaulting to `"debian"`.
/// Read the current vendor name from the active dpkg origins `default` file
/// (`$DPKG_ORIGINS_DIR/default`, falling back to `/etc/dpkg/origins/default`;
/// its `Vendor:` or `Origin:` field), defaulting to `"debian"`.
pub fn current_vendor() -> String {
read_vendor_from(Path::new("/etc/dpkg/origins/default")).unwrap_or_else(|| "debian".to_string())
let path = resolve_origins_default(
std::env::var("DPKG_ORIGINS_DIR").ok().as_deref(),
"/etc/dpkg/origins",
);
std::fs::read_to_string(path)
.ok()
.and_then(|content| vendor_from_origins_content(&content))
.unwrap_or_else(|| "debian".to_string())
}
fn read_vendor_from(path: &Path) -> Option<String> {
let content = std::fs::read_to_string(path).ok()?;
/// Resolve the path of the active dpkg origins file from the
/// `DPKG_ORIGINS_DIR` value (the directory holding the origin files, where
/// `default` selects the active one) and the fallback directory
/// (`/etc/dpkg/origins`). An unset or empty directory value falls back.
fn resolve_origins_default(origins_dir: Option<&str>, fallback_dir: &str) -> PathBuf {
let dir = origins_dir
.filter(|d| !d.is_empty())
.unwrap_or(fallback_dir);
Path::new(dir).join("default")
}
/// Extract the vendor name from the content of a dpkg origins file: its
/// `Vendor:` field, falling back to `Origin:` when absent. `None` when
/// neither field carries a non-empty value.
pub fn vendor_from_origins_content(content: &str) -> Option<String> {
for line in content.lines() {
if let Some(value) = line.strip_prefix("Vendor:") {
let v = value.trim();
@@ -87,15 +135,15 @@ fn read_vendor_from(path: &Path) -> Option<String> {
/// Default build profiles applied by vendor hooks.
///
/// The Ubuntu vendor module activates `derivative.ubuntu noudeb` by default;
/// Debian applies none. This mirrors what `Dpkg::BuildProfiles` resolves when
/// `DEB_BUILD_PROFILES` is unset.
/// The distro data carries them (`build_profiles` of the vendor's
/// distribution in `data/distro_info.yml` — the Ubuntu vendor activates
/// `derivative.ubuntu noudeb`, Debian applies none), mirroring what
/// `Dpkg::BuildProfiles` resolves when `DEB_BUILD_PROFILES` is unset. The
/// vendor is matched case-insensitively against the distro data keys
/// (dpkg's `Vendor:` field keeps its original casing); a vendor with no
/// distro entry gets no profiles.
pub fn default_build_profiles(vendor: &str) -> Vec<String> {
if vendor.eq_ignore_ascii_case("ubuntu") {
vec!["derivative.ubuntu".to_string(), "noudeb".to_string()]
} else {
Vec::new()
}
crate::distro_info::get_build_profiles(&vendor.to_lowercase()).unwrap_or_default()
}
/// Resolve the active build profiles: explicit `-P` profiles take precedence,
@@ -266,13 +314,63 @@ mod tests {
assert_eq!(env.get("LANG").unwrap(), "C");
assert_eq!(env.get("LC_ALL").unwrap(), "C");
assert_eq!(env.get("SOURCE_DATE_EPOCH").unwrap(), "1787392800");
assert_eq!(env.get("DEB_BUILD_OPTIONS").unwrap(), "parallel=16");
// Reading the var is race-free; the expected value goes through the
// same merge so the assertion holds whatever the ambient environment
// carries.
let expected = merge_deb_build_options(
std::env::var("DEB_BUILD_OPTIONS").ok().as_deref(),
"parallel=16",
);
assert_eq!(env.get("DEB_BUILD_OPTIONS").unwrap(), &expected);
assert!(!env.contains_key("DEB_BUILD_PROFILES"));
let env = build_env(1, 4, &["nodoc".to_string(), "cross".to_string()]);
assert_eq!(env.get("DEB_BUILD_PROFILES").unwrap(), "nodoc,cross");
}
/// dpkg-buildpackage prepends the inherited `DEB_BUILD_OPTIONS`, so
/// user-set options survive alongside the computed ones.
#[test]
fn merge_prepends_inherited_options() {
assert_eq!(
merge_deb_build_options(Some("terse"), "parallel=16"),
"terse parallel=16"
);
assert_eq!(
merge_deb_build_options(Some("nocheck terse"), "parallel=4"),
"nocheck terse parallel=4"
);
}
/// An unset, empty or blank inherited value yields just the computed
/// options.
#[test]
fn merge_skips_empty_inherited() {
assert_eq!(merge_deb_build_options(None, "parallel=8"), "parallel=8");
assert_eq!(
merge_deb_build_options(Some(""), "parallel=8"),
"parallel=8"
);
assert_eq!(
merge_deb_build_options(Some(" "), "parallel=8"),
"parallel=8"
);
}
/// Both sides are trimmed and internal whitespace runs collapsed: no
/// leading/trailing space, no double spaces in the merged result.
#[test]
fn merge_normalizes_whitespace() {
assert_eq!(
merge_deb_build_options(Some(" terse "), "parallel=2"),
"terse parallel=2"
);
assert_eq!(
merge_deb_build_options(Some("nocheck\t terse"), "parallel=2"),
"nocheck terse parallel=2"
);
}
#[test]
fn vendor_defaults() {
assert!(default_build_profiles("debian").is_empty());
@@ -280,6 +378,53 @@ mod tests {
default_build_profiles("ubuntu"),
vec!["derivative.ubuntu".to_string(), "noudeb".to_string()]
);
// dpkg's Vendor field keeps its original casing; the distro data
// keys are lowercase.
assert_eq!(
default_build_profiles("Ubuntu"),
vec!["derivative.ubuntu".to_string(), "noudeb".to_string()]
);
// A vendor without a distro entry gets no profiles.
assert!(default_build_profiles("some-derivative").is_empty());
}
#[test]
fn vendor_from_origins_content_prefers_vendor_then_origin() {
assert_eq!(
vendor_from_origins_content("Vendor: Ubuntu\nSuite: noble\n"),
Some("Ubuntu".to_string())
);
// Origin fallback when no Vendor field is present.
assert_eq!(
vendor_from_origins_content("Origin: Debian\nSuite: stable\n"),
Some("Debian".to_string())
);
// Empty Vendor falls through to Origin.
assert_eq!(
vendor_from_origins_content("Vendor: \nOrigin: Debian\n"),
Some("Debian".to_string())
);
assert_eq!(vendor_from_origins_content("Suite: stable\n"), None);
}
/// `current_vendor` must honor `DPKG_ORIGINS_DIR` (already on the
/// `.buildinfo` allow-list) when locating the `default` origins file,
/// falling back to `/etc/dpkg/origins/default` when unset or empty.
#[test]
fn origins_default_path_honors_dpkg_origins_dir() {
assert_eq!(
resolve_origins_default(Some("/custom/origins"), "/etc/dpkg/origins"),
PathBuf::from("/custom/origins/default")
);
assert_eq!(
resolve_origins_default(None, "/etc/dpkg/origins"),
PathBuf::from("/etc/dpkg/origins/default")
);
// An empty value behaves as unset, like dpkg's `$dir || $default`.
assert_eq!(
resolve_origins_default(Some(""), "/etc/dpkg/origins"),
PathBuf::from("/etc/dpkg/origins/default")
);
}
#[test]
+1025 -235
View File
File diff suppressed because it is too large Load Diff
+1093 -79
View File
File diff suppressed because it is too large Load Diff
+199 -17
View File
@@ -70,6 +70,12 @@ pub trait ContextDriver {
fn read_file(&self, path: &Path) -> io::Result<String>;
fn write_file(&self, path: &Path, content: &str) -> io::Result<()>;
fn exists(&self, path: &Path) -> io::Result<bool>;
/// Check if a path is a directory inside the context
///
/// Distinct from [`ContextDriver::exists`] because paths returned by
/// [`ContextDriver::list_files`] are context-relative and can only be
/// classified through the context, never with a host-side stat.
fn is_dir(&self, path: &Path) -> io::Result<bool>;
/// Clean up any resources held by the driver (e.g. unmount overlay filesystems).
/// Called before the chroot directory is removed.
@@ -117,6 +123,28 @@ pub enum ContextConfig {
},
}
impl ContextConfig {
/// Build an SSH context configuration from an endpoint of the form
/// `[ssh://][user@]host[:port]`.
pub fn from_endpoint(endpoint: &str) -> Result<Self, String> {
let re = regex::Regex::new(
r"^(?:ssh://)?(?:(?P<user>[^@]+)@)?(?P<host>[^:/]+)(?::(?P<port>\d+))?$",
)
.expect("valid endpoint regex");
let cap = re.captures(endpoint).ok_or_else(|| {
format!("Invalid endpoint format: '{endpoint}'. Expected [ssh://][user@]host[:port]")
})?;
let host = cap.name("host").unwrap().as_str().to_string();
let user = cap.name("user").map(|m| m.as_str().to_string());
let port = cap
.name("port")
.map(|m| m.as_str().parse::<u16>())
.transpose()
.map_err(|_| "Invalid port number".to_string())?;
Ok(ContextConfig::Ssh { host, user, port })
}
}
/// A context, allowing to run commands, read and write files, etc
pub struct Context {
/// Configuration for the context
@@ -131,8 +159,46 @@ pub struct Context {
impl Context {
/// Create a context from configuration
pub fn new(config: ContextConfig) -> Self {
let parent = match &config {
///
/// Parent contexts named in the configuration are resolved through the
/// global context manager; a dangling parent name is reported as an
/// error instead of panicking.
///
/// Note that this takes a read lock on the global manager's
/// configuration: never call it while holding that lock for writing.
/// [`crate::context::ContextManager`] itself goes through
/// [`Context::with_lookup`] instead, which takes no locks.
pub fn new(config: ContextConfig) -> io::Result<Self> {
Self::with_lookup(config, &|name| {
crate::context::manager::MANAGER
.get_config()
.contexts
.get(name)
.cloned()
})
}
/// Create a context from configuration, resolving `parent` context names
/// through `lookup` instead of the global context manager.
///
/// `lookup` must be lock-free: this is what allows
/// [`crate::context::ContextManager`] to build contexts while holding
/// (or before the very existence of) its configuration lock. Returns an
/// error when a referenced parent does not exist or when the parent
/// chain contains a cycle.
pub(crate) fn with_lookup(
config: ContextConfig,
lookup: &dyn Fn(&str) -> Option<ContextConfig>,
) -> io::Result<Self> {
Self::with_lookup_inner(config, lookup, &mut Vec::new())
}
fn with_lookup_inner(
config: ContextConfig,
lookup: &dyn Fn(&str) -> Option<ContextConfig>,
chain: &mut Vec<String>,
) -> io::Result<Self> {
let parent_name = match &config {
ContextConfig::Schroot {
parent: Some(parent_name),
..
@@ -140,23 +206,37 @@ impl Context {
| ContextConfig::Unshare {
parent: Some(parent_name),
..
} => {
let config_lock = crate::context::manager::MANAGER.get_config();
let parent_config = config_lock
.contexts
.get(parent_name)
.cloned()
.expect("Parent context not found");
Some(Arc::new(Context::new(parent_config)))
} => parent_name.clone(),
_ => {
return Ok(Self {
config,
parent: None,
driver: Mutex::new(None),
});
}
_ => None,
};
Self {
config,
parent,
driver: Mutex::new(None),
if chain.iter().any(|name| name == &parent_name) {
return Err(io::Error::new(
io::ErrorKind::InvalidData,
format!("Parent context cycle: '{parent_name}' appears in its own parent chain"),
));
}
let parent_config = lookup(&parent_name).ok_or_else(|| {
io::Error::new(
io::ErrorKind::NotFound,
format!("Parent context '{parent_name}' not found"),
)
})?;
chain.push(parent_name);
let parent = Self::with_lookup_inner(parent_config, lookup, chain);
chain.pop();
Ok(Self {
config,
parent: Some(Arc::new(parent?)),
driver: Mutex::new(None),
})
}
/// Create a context with an explicit parent context
@@ -169,12 +249,22 @@ impl Context {
}
/// Make a command inside context
///
/// Build tooling must not inherit the session's locale: dpkg-family
/// tools and perl-based packaging scripts change their output (and
/// dpkg-buildpackage treats some of it as data) with the environment,
/// and a translated or mixed locale leaks host state into builds. The
/// C locale is the default; a caller can still override it by setting
/// LANG/LC_ALL through [`ContextCommand::envs`] afterwards.
pub fn command<S: AsRef<OsStr>>(&self, program: S) -> ContextCommand<'_> {
ContextCommand {
context: self,
program: program.as_ref().to_string_lossy().to_string(),
args: Vec::new(),
env: Vec::new(),
env: vec![
("LANG".to_string(), "C".to_string()),
("LC_ALL".to_string(), "C".to_string()),
],
cwd: None,
sink: None,
}
@@ -229,6 +319,15 @@ impl Context {
self.driver().as_ref().unwrap().exists(path)
}
/// Check if a path is a directory inside context
///
/// Paths returned by [`Context::list_files`] are context-relative
/// (e.g. rooted inside the chroot for an unshare context): whether they
/// are directories can only be decided through the context.
pub fn is_dir(&self, path: &Path) -> io::Result<bool> {
self.driver().as_ref().unwrap().is_dir(path)
}
/// Clean up any resources held by the driver (e.g. unmount overlay filesystems).
/// Called before the chroot directory is removed.
pub fn cleanup(&self) -> io::Result<()> {
@@ -259,7 +358,9 @@ impl Context {
overlay_mounts: std::sync::Mutex::new(Vec::new()),
}),
};
*driver_lock = Some(driver);
// In test runs, commands whose output would inherit the terminal
// are captured into the per-test log file instead
*driver_lock = Some(crate::test_support::wrap_driver(driver));
}
driver_lock
}
@@ -401,3 +502,84 @@ fn contextualize_spawn_error(program: &str, e: io::Error) -> io::Error {
io::Error::new(e.kind(), format!("Could not run '{program}': {e}"))
}
}
#[cfg(test)]
mod endpoint_tests {
use super::*;
/// Every accepted endpoint spelling maps to the expected config.
#[test]
fn from_endpoint_parses_all_spellings() {
assert_eq!(
ContextConfig::from_endpoint("myhost"),
Ok(ContextConfig::Ssh {
host: "myhost".into(),
user: None,
port: None,
})
);
assert_eq!(
ContextConfig::from_endpoint("admin@myhost"),
Ok(ContextConfig::Ssh {
host: "myhost".into(),
user: Some("admin".into()),
port: None,
})
);
assert_eq!(
ContextConfig::from_endpoint("myhost:2222"),
Ok(ContextConfig::Ssh {
host: "myhost".into(),
user: None,
port: Some(2222),
})
);
assert_eq!(
ContextConfig::from_endpoint("ssh://admin@myhost:22"),
Ok(ContextConfig::Ssh {
host: "myhost".into(),
user: Some("admin".into()),
port: Some(22),
})
);
}
/// Non-numeric ports and extra segments are format errors; a
/// non-u16 numeric port is a port error.
#[test]
fn from_endpoint_rejects_malformed_endpoints() {
for bad in ["", "a/b/c", "host:notaport"] {
let err = ContextConfig::from_endpoint(bad).unwrap_err();
assert!(err.contains("Invalid endpoint format"), "{err}");
}
let err = ContextConfig::from_endpoint("host:99999").unwrap_err();
assert_eq!(err, "Invalid port number");
}
/// Commands run in the C locale whatever the session environment
/// carries: host locale variables must not leak into builds. An
/// explicit caller override still wins.
#[test]
fn commands_default_to_the_c_locale() {
let ctx = Context::new(ContextConfig::Local).unwrap();
let locale = ctx
.command("sh")
.arg("-c")
.arg("printf '%s' \"${LC_ALL:-unset}:${LANG:-unset}\"")
.output()
.unwrap()
.stdout;
assert_eq!(String::from_utf8_lossy(&locale), "C:C");
let locale = ctx
.command("sh")
.arg("-c")
.arg("printf '%s' \"$LC_ALL\"")
.env("LC_ALL", "C.UTF-8")
.output()
.unwrap()
.stdout;
assert_eq!(String::from_utf8_lossy(&locale), "C.UTF-8");
}
}
+52 -14
View File
@@ -24,33 +24,31 @@ impl ContextDriver for LocalDriver {
}
fn create_temp_dir(&self) -> io::Result<String> {
// Generate a unique temporary directory name with random string
// Sub-second precision and an atomic create: two concurrent
// contexts racing on the same name must never share a directory,
// so the loser of a create falls through to the next attempt
// instead of probing for existence first (a probe-then-create
// window loses exactly when two callers arrive together).
let base_timestamp = SystemTime::now()
.duration_since(SystemTime::UNIX_EPOCH)
.unwrap()
.as_secs();
.as_millis();
let mut attempt = 0;
loop {
let work_dir_name = if attempt == 0 {
format!("pkh-{}", base_timestamp)
format!("pkh-{base_timestamp}")
} else {
format!("pkh-{}-{}", base_timestamp, attempt)
format!("pkh-{base_timestamp}-{attempt}")
};
let temp_dir_path = std::env::temp_dir().join(&work_dir_name);
// Check if directory already exists
if temp_dir_path.exists() {
attempt += 1;
continue;
match std::fs::create_dir(&temp_dir_path) {
Ok(()) => return Ok(temp_dir_path.to_string_lossy().to_string()),
Err(e) if e.kind() == io::ErrorKind::AlreadyExists => attempt += 1,
Err(e) => return Err(e),
}
// Create the directory
std::fs::create_dir_all(&temp_dir_path)?;
// Return the path as a string
return Ok(temp_dir_path.to_string_lossy().to_string());
}
}
@@ -157,6 +155,10 @@ impl ContextDriver for LocalDriver {
fn exists(&self, path: &Path) -> io::Result<bool> {
Ok(path.exists())
}
fn is_dir(&self, path: &Path) -> io::Result<bool> {
Ok(path.is_dir())
}
}
fn copy_dir_recursive(src: &Path, dest: &Path) -> io::Result<()> {
@@ -190,3 +192,39 @@ fn copy_dir_recursive(src: &Path, dest: &Path) -> io::Result<()> {
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
/// Concurrent callers must never share a temporary directory: the
/// create is atomic, so a lost race falls through to the next name
/// instead of both callers probing the same free name and unpacking
/// into the same directory.
#[test]
fn create_temp_dir_is_unique_under_concurrency() {
const CALLERS: usize = 8;
let (tx, rx) = std::sync::mpsc::channel();
let handles: Vec<_> = (0..CALLERS)
.map(|_| {
let tx = tx.clone();
std::thread::spawn(move || {
let dir = LocalDriver.create_temp_dir().unwrap();
tx.send(dir).unwrap();
})
})
.collect();
for handle in handles {
handle.join().unwrap();
}
drop(tx);
let mut names: Vec<String> = rx.iter().collect();
names.sort();
let unique: std::collections::BTreeSet<&String> = names.iter().collect();
assert_eq!(names.len(), unique.len(), "duplicate temp dirs: {names:?}");
for name in &unique {
std::fs::remove_dir(name).unwrap();
}
}
}
+144 -36
View File
@@ -3,7 +3,7 @@ use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::fs;
use std::io;
use std::path::PathBuf;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::RwLock;
@@ -48,26 +48,88 @@ impl ContextManager {
fs::create_dir_all(config_dir)?;
let config_path = config_dir.join("contexts.json");
let config = if config_path.exists() {
// Load existing configuration file
let content = fs::read_to_string(&config_path)?;
serde_json::from_str(&content)
.map_err(|e| io::Error::new(io::ErrorKind::InvalidData, e))?
} else {
// Create a new configuration file
Config::default()
let mut config = Self::load_config(&config_path);
// Build the initial Context against the freshly loaded map, before
// the manager itself exists: resolution must not go through the
// global MANAGER here, since a parented current context would
// re-enter its own LazyLock initialization.
let initial = match Self::make_context(&config.context, &config.contexts) {
Ok(context) => context,
Err(e) => {
log::error!(
"Cannot build current context '{}' from {}: {e}; falling back to 'local'",
config.context,
config_path.display()
);
config.context = "local".to_string();
Self::make_context("local", &config.contexts).unwrap_or_else(|e| {
// Only possible in a hand-edited configuration without
// any 'local' entry; a plain Local context has no parent
// and cannot fail to build.
log::error!(
"'local' context missing from {}: {e}",
config_path.display()
);
Context::new(ContextConfig::Local).expect("Local context cannot fail")
})
}
};
Ok(Self {
context: RwLock::new(Arc::new(Self::make_context(
config.context.as_str(),
&config,
))),
context: RwLock::new(Arc::new(initial)),
config_path,
config: RwLock::new(config),
})
}
/// Load the configuration stored at `path`.
///
/// A missing file yields [`Config::default`]. A file that cannot be read
/// or parsed must not take the whole program down: this falls back to
/// the default (local-only) configuration and logs an error. Because a
/// later [`ContextManager::save`] would otherwise silently overwrite the
/// corrupt file and destroy its content, the corrupt file is first
/// backed up to `<path>.bak` (best effort).
pub(crate) fn load_config(path: &Path) -> Config {
if !path.exists() {
return Config::default();
}
let loaded = fs::read_to_string(path).and_then(|content| {
serde_json::from_str(&content)
.map_err(|e| io::Error::new(io::ErrorKind::InvalidData, e))
});
match loaded {
Ok(config) => config,
Err(e) => {
log::error!(
"Context configuration {} is corrupt ({e}); using the default (local-only) configuration",
path.display()
);
Self::backup_corrupt_file(path);
Config::default()
}
}
}
/// Back up a corrupt configuration file (best effort) so a later save
/// cannot silently destroy its content.
fn backup_corrupt_file(path: &Path) {
let mut os = path.as_os_str().to_os_string();
os.push(".bak");
let backup_path = PathBuf::from(os);
match fs::copy(path, &backup_path) {
Ok(_) => log::warn!(
"Corrupt context configuration backed up to {}",
backup_path.display()
),
Err(e) => log::warn!(
"Could not back up corrupt context configuration to {}: {e}",
backup_path.display()
),
}
}
/// Obtain current ContextManager configuration
pub fn get_config(&self) -> std::sync::RwLockReadGuard<'_, Config> {
self.config.read().unwrap()
@@ -77,7 +139,12 @@ impl ContextManager {
pub fn with_path(path: PathBuf) -> Self {
let config = Config::default();
Self {
context: RwLock::new(Arc::new(Self::make_context("local", &config))),
// 'local' is always present in Config::default and has no
// parent, so this cannot fail.
context: RwLock::new(Arc::new(
Self::make_context("local", &config.contexts)
.expect("default 'local' context cannot fail"),
)),
config_path: path,
config: RwLock::new(config),
}
@@ -92,13 +159,22 @@ impl ContextManager {
Ok(())
}
fn make_context(name: &str, config: &Config) -> Context {
let context_config = config
.contexts
.get(name)
.cloned()
.expect("Context not found in config");
Context::new(context_config)
/// Build a [`Context`] for `name` from `contexts`.
///
/// Lock-free by construction: parent references are resolved against
/// `contexts` itself (see [`Context::with_lookup`]), never against the
/// manager's configuration lock. This is what keeps [`ContextManager::new`]
/// working before the global [`MANAGER`] exists, and what allows callers
/// to build contexts without risking a re-entrant read on a lock they
/// already hold for writing.
fn make_context(name: &str, contexts: &HashMap<String, ContextConfig>) -> io::Result<Context> {
let context_config = contexts.get(name).cloned().ok_or_else(|| {
io::Error::new(
io::ErrorKind::NotFound,
format!("Context '{name}' not found in configuration"),
)
})?;
Context::with_lookup(context_config, &|parent| contexts.get(parent).cloned())
}
/// List contexts from configuration
@@ -124,45 +200,77 @@ impl ContextManager {
/// Remove context from configuration
pub fn remove_context(&self, name: &str) -> io::Result<()> {
let mut config = self.config.write().unwrap();
if name == "local" {
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
"Cannot remove local context",
));
}
if config.contexts.remove(name).is_some() {
// If we are removing the current context, fallback to local
// Mutate under the write lock, snapshotting the remaining map when
// the removed context was current; the fallback Context is built
// after the lock is released (same discipline as `set_current`).
let fallback_contexts = {
let mut config = self.config.write().unwrap();
if config.contexts.remove(name).is_none() {
return Ok(());
}
if name == config.context {
// If we are removing the current context, fallback to local
config.context = "local".to_string();
self.set_current_ephemeral(Self::make_context("local", &config));
Some(config.contexts.clone())
} else {
None
}
};
drop(config); // Drop write lock before saving
self.save()?;
if let Some(contexts) = fallback_contexts {
self.set_current_ephemeral(Self::make_context("local", &contexts)?);
}
self.save()?;
Ok(())
}
/// Set current context from name (modifying configuration)
pub fn set_current(&self, name: &str) -> io::Result<()> {
// Snapshot what `make_context` needs and release the lock before
// building the Context. Building resolves parent contexts, and this
// code path used to hold the config write guard while re-entering
// the same lock for a read — a guaranteed deadlock on a
// std::sync::RwLock (same-thread write-then-read).
let contexts = self.config.read().unwrap().contexts.clone();
if !contexts.contains_key(name) {
return Err(io::Error::new(
io::ErrorKind::NotFound,
format!("Context '{name}' not found"),
));
}
let context = Self::make_context(name, &contexts)?;
// Re-take the write lock briefly to commit. The name may have been
// removed between snapshot and commit; report the same NotFound
// error instead of persisting a dangling current-context reference.
let mut config = self.config.write().unwrap();
if config.contexts.contains_key(name) {
if !config.contexts.contains_key(name) {
return Err(io::Error::new(
io::ErrorKind::NotFound,
format!("Context '{name}' not found"),
));
}
config.context = name.to_string();
self.set_current_ephemeral(Self::make_context(name, &config));
drop(config); // Drop write lock before saving
self.set_current_ephemeral(context);
self.save()?;
Ok(())
} else {
Err(io::Error::new(
io::ErrorKind::NotFound,
format!("Context '{}' not found", name),
))
}
}
/// Set current context, without modifying configuration
pub fn set_current_ephemeral(&self, context: Context) {
///
/// Accepts either an owned [`Context`] or an already-shared
/// `Arc<Context>`: callers that keep their own handle to the context
/// they install (e.g. [`crate::deb::ephemeral::EphemeralContextGuard`])
/// pass the Arc so they can restore exactly this context afterwards
/// instead of relying on whatever happens to be current at that time.
pub fn set_current_ephemeral(&self, context: impl Into<Arc<Context>>) {
*self.context.write().unwrap() = context.into();
}
+180 -4
View File
@@ -3,10 +3,15 @@ pub(crate) mod capture;
mod local;
mod manager;
mod schroot;
pub(crate) mod shell;
mod ssh;
mod unshare;
pub use api::{Context, ContextCommand, ContextConfig, LineSink, Stream};
// The driver trait is implementation detail of the context API; it is only
// needed crate-internally (test-run capture wrapper), so keep it out of the
// public surface (and its documentation requirement).
pub(crate) use api::ContextDriver;
pub use manager::ContextManager;
use std::sync::Arc;
@@ -75,7 +80,7 @@ mod tests {
let src_file = temp_dir.path().join("src.txt");
fs::write(&src_file, "local").unwrap();
let ctx = Context::new(ContextConfig::Local);
let ctx = Context::new(ContextConfig::Local).unwrap();
let dest = ctx.ensure_available(&src_file, "/tmp").unwrap();
// Should return a path that exists and has the same content
@@ -154,10 +159,147 @@ mod tests {
assert!(mgr.list_contexts().contains(&"local".to_string()));
}
/// `set_current` on a context whose configuration carries a `parent`
/// must complete without deadlocking: building the Context resolves the
/// parent chain, which used to re-enter the config lock while
/// `set_current` still held it for writing (a guaranteed deadlock on a
/// std::sync::RwLock, same-thread write-then-read).
#[test]
fn test_set_current_parented_context_no_deadlock() {
let temp_file = NamedTempFile::new().unwrap();
let mgr = Arc::new(ContextManager::with_path(temp_file.path().to_path_buf()));
mgr.add_context("base", ContextConfig::Local).unwrap();
mgr.add_context(
"child",
ContextConfig::Schroot {
name: "testchroot".to_string(),
parent: Some("base".to_string()),
},
)
.unwrap();
// Run with a timeout so a regression fails fast instead of hanging
// the test binary forever.
let (tx, rx) = std::sync::mpsc::channel();
let worker = {
let mgr = mgr.clone();
std::thread::spawn(move || {
let result = mgr.set_current("child");
tx.send(()).expect("receiver still waiting");
result
})
};
match rx.recv_timeout(std::time::Duration::from_secs(30)) {
Ok(()) => {}
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {
panic!("set_current() deadlocked building a parented context");
}
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => {
panic!("set_current() thread panicked before completing");
}
}
worker.join().unwrap().unwrap();
assert_eq!(mgr.current_name(), "child");
}
/// A context referencing a missing parent must produce an error, not a
/// panic (the parent lookup used to `.expect()`).
#[test]
fn test_set_current_dangling_parent_errors() {
let temp_file = NamedTempFile::new().unwrap();
let mgr = ContextManager::with_path(temp_file.path().to_path_buf());
mgr.add_context(
"orphan",
ContextConfig::Unshare {
path: "/some/chroot".to_string(),
parent: Some("missing".to_string()),
},
)
.unwrap();
let err = mgr.set_current("orphan").unwrap_err();
assert_eq!(err.kind(), std::io::ErrorKind::NotFound);
// Nothing was committed: the current context is unchanged.
assert_eq!(mgr.current_name(), "local");
}
/// A parent cycle in a hand-edited configuration must be rejected with
/// an error instead of recursing until the stack overflows.
#[test]
fn test_set_current_parent_cycle_errors() {
let temp_file = NamedTempFile::new().unwrap();
let mgr = ContextManager::with_path(temp_file.path().to_path_buf());
mgr.add_context(
"a",
ContextConfig::Schroot {
name: "schroot-a".to_string(),
parent: Some("b".to_string()),
},
)
.unwrap();
mgr.add_context(
"b",
ContextConfig::Unshare {
path: "/chroot-b".to_string(),
parent: Some("a".to_string()),
},
)
.unwrap();
let err = mgr.set_current("a").unwrap_err();
assert_eq!(err.kind(), std::io::ErrorKind::InvalidData);
}
/// A corrupt contexts.json must not take the manager down:
/// `load_config` falls back to the default (local-only) configuration,
/// keeps the corrupt file in place and backs it up to contexts.json.bak
/// so a later save cannot silently destroy its content.
#[test]
fn test_load_config_corrupt_file_falls_back_and_backs_up() {
let temp_dir = tempfile::tempdir().unwrap();
let config_path = temp_dir.path().join("contexts.json");
let garbage = "{ this is definitely not valid json";
fs::write(&config_path, garbage).unwrap();
let config = ContextManager::load_config(&config_path);
// Falls back to the default (local-only) configuration...
assert_eq!(config.context, "local");
assert!(config.contexts.contains_key("local"));
// ...preserving the corrupt file via the backup, original untouched.
let backup_path = temp_dir.path().join("contexts.json.bak");
assert_eq!(fs::read_to_string(&backup_path).unwrap(), garbage);
assert_eq!(fs::read_to_string(&config_path).unwrap(), garbage);
// A subsequent save replaces only the original, never the backup.
let mgr = ContextManager::with_path(config_path.clone());
mgr.add_context("newctx", ContextConfig::Local).unwrap();
let rewritten = fs::read_to_string(&config_path).unwrap();
serde_json::from_str::<super::manager::Config>(&rewritten).unwrap();
assert_eq!(fs::read_to_string(&backup_path).unwrap(), garbage);
}
/// A missing contexts.json yields the default configuration and writes
/// nothing (no file, no backup) until an explicit save.
#[test]
fn test_load_config_missing_file_defaults() {
let temp_dir = tempfile::tempdir().unwrap();
let config_path = temp_dir.path().join("contexts.json");
let config = ContextManager::load_config(&config_path);
assert_eq!(config.context, "local");
assert!(config.contexts.contains_key("local"));
assert!(!config_path.exists());
assert!(!temp_dir.path().join("contexts.json.bak").exists());
}
#[test]
fn test_context_file_ops() {
let temp_dir = tempfile::tempdir().unwrap();
let ctx = Context::new(ContextConfig::Local);
let ctx = Context::new(ContextConfig::Local).unwrap();
let file_path = temp_dir.path().join("test.txt");
let content = "hello world";
@@ -199,7 +341,7 @@ mod tests {
fn test_context_copy_preserves_dangling_symlink() {
use std::os::unix::fs::symlink;
let temp_dir = tempfile::tempdir().unwrap();
let ctx = Context::new(ContextConfig::Local);
let ctx = Context::new(ContextConfig::Local).unwrap();
let src_dir = temp_dir.path().join("src");
std::fs::create_dir_all(&src_dir).unwrap();
@@ -235,7 +377,7 @@ mod tests {
fs::create_dir_all(src_root.join("src/.svn")).unwrap();
fs::write(src_root.join("src/hello.c"), "int main() {}").unwrap();
let ctx = Context::new(ContextConfig::Local);
let ctx = Context::new(ContextConfig::Local).unwrap();
let dest = ctx.ensure_available(&src_root, "/tmp").unwrap();
assert!(dest.join("src/hello.c").exists());
@@ -243,6 +385,40 @@ mod tests {
assert!(!dest.join("src/.svn").exists());
}
/// The unshare driver maps context-relative paths onto the chroot root
/// on the host: `is_dir` must answer through that mapping (a host-side
/// stat of the unmapped path sees nothing), which is what lets the deb
/// package-directory search classify staged entries.
#[test]
fn test_unshare_is_dir_maps_through_the_chroot_root() {
let chroot = tempfile::tempdir().unwrap();
fs::create_dir_all(chroot.path().join("tmp/work/tree/debian")).unwrap();
fs::write(chroot.path().join("tmp/work/orig.tar.xz"), "tar").unwrap();
let base = Context::new(ContextConfig::Local).unwrap();
let ctx = Context::with_parent(
ContextConfig::Unshare {
path: chroot.path().to_string_lossy().to_string(),
parent: None,
},
Arc::new(base),
);
assert!(ctx.is_dir(std::path::Path::new("/tmp/work/tree")).unwrap());
assert!(
ctx.exists(std::path::Path::new("/tmp/work/tree/debian"))
.unwrap()
);
assert!(
!ctx.is_dir(std::path::Path::new("/tmp/work/orig.tar.xz"))
.unwrap()
);
assert!(
!ctx.exists(std::path::Path::new("/tmp/work/missing"))
.unwrap()
);
}
/// The overlay-mount path exposes the tree verbatim, so pruning happens
/// after the fact: nested VCS metadata must be removed recursively.
#[test]
+106 -12
View File
@@ -1,6 +1,7 @@
/// Schroot context: execute commands in a schroot session
/// Not tested, will need more work!
use super::api::{ContextDriver, LineSink};
use super::api::{Context, ContextConfig, ContextDriver, LineSink};
use super::shell::shell_quote;
use std::io;
use std::path::{Path, PathBuf};
use std::sync::Arc;
@@ -11,13 +12,12 @@ pub struct SchrootDriver {
pub parent: Option<Arc<super::api::Context>>,
}
use super::api::{Context, ContextConfig};
impl SchrootDriver {
fn parent(&self) -> Arc<Context> {
self.parent
.clone()
.unwrap_or_else(|| Arc::new(Context::new(ContextConfig::Local)))
self.parent.clone().unwrap_or_else(|| {
// ContextConfig::Local has no parent, so this cannot fail.
Arc::new(Context::new(ContextConfig::Local).expect("Local context cannot fail"))
})
}
fn ensure_session(&self) -> io::Result<String> {
@@ -106,6 +106,11 @@ impl SchrootDriver {
/// Wrap `(program, args)` in `sh -c` when a working directory or
/// environment variables are needed.
///
/// Everything interpolated into the resulting shell string — the `cd`
/// target, env keys and values, the program and each argument — is
/// POSIX-shell-quoted (see [`shell_quote`]), so metacharacters (spaces,
/// quotes, `$`, ...) can neither split words nor trigger expansion.
fn wrap_command(
program: &str,
args: &[String],
@@ -119,17 +124,21 @@ impl SchrootDriver {
let mut shell_cmd = String::new();
if let Some(dir) = cwd {
shell_cmd.push_str(&format!("cd {} && ", dir));
shell_cmd.push_str(&format!("cd {} && ", shell_quote(dir)));
}
if !env.is_empty() {
shell_cmd.push_str("env ");
for (k, v) in env {
shell_cmd.push_str(&format!("{}={} ", k, v));
shell_cmd.push_str(&format!("{}={} ", shell_quote(k), shell_quote(v)));
}
}
shell_cmd.push_str(&format!("{} {}", program, args.join(" ")));
shell_cmd.push_str(&shell_quote(program));
for arg in args {
shell_cmd.push(' ');
shell_cmd.push_str(&shell_quote(arg));
}
actual_program = "sh".to_string();
actual_args = vec!["-c".to_string(), shell_cmd];
@@ -260,9 +269,13 @@ impl ContextDriver for SchrootDriver {
&[
"-c".to_string(),
format!(
"echo -ne '{}' > '{}'",
content.replace("'", "'\\''"),
path.to_string_lossy()
// `printf '%s'` writes the content verbatim (the previous
// `echo -ne` mangled backslashes, and dash's echo prints
// "-ne" literally). Content and path are shell-quoted so
// metacharacters in either cannot break out.
"printf '%s' {} > {}",
shell_quote(content),
shell_quote(&path.to_string_lossy())
),
],
&[],
@@ -283,4 +296,85 @@ impl ContextDriver for SchrootDriver {
)?;
Ok(status.success())
}
fn is_dir(&self, path: &Path) -> io::Result<bool> {
let status = self.run(
"test",
&["-d".to_string(), path.to_string_lossy().to_string()],
&[],
None,
)?;
Ok(status.success())
}
}
#[cfg(test)]
mod tests {
use super::SchrootDriver;
/// Without cwd/env the program and args go to schroot as direct argv
/// (no shell involved), so they must pass through untouched.
#[test]
fn wrap_command_passthrough_without_env_or_cwd() {
let (prog, args) = SchrootDriver::wrap_command(
"make",
&["install".to_string(), "DEST=x y".to_string()],
&[],
None,
);
assert_eq!(prog, "make");
assert_eq!(args, vec!["install".to_string(), "DEST=x y".to_string()]);
}
/// A value with a space must stay a single env assignment: previously
/// DEB_BUILD_OPTIONS="parallel=4 nocheck" made sh treat `nocheck` as
/// the command to run.
#[test]
fn wrap_command_quotes_env_values_cwd_and_args() {
let (prog, args) = SchrootDriver::wrap_command(
"dpkg-buildpackage",
&["-us".to_string(), "-uc".to_string()],
&[(
"DEB_BUILD_OPTIONS".to_string(),
"parallel=4 nocheck".to_string(),
)],
Some("/build/pkg 1.0"),
);
assert_eq!(prog, "sh");
assert_eq!(args[0], "-c");
assert_eq!(
args[1],
"cd '/build/pkg 1.0' && env 'DEB_BUILD_OPTIONS'='parallel=4 nocheck' \
'dpkg-buildpackage' '-us' '-uc'"
);
}
/// The definitive check: a real shell must execute the wrapped command
/// exactly as intended — cwd applied, env set verbatim, the inner
/// program invoked with its argument — despite quotes in the values.
#[cfg(unix)]
#[test]
fn wrapped_command_survives_shell_parsing() {
let dir = tempfile::tempdir().unwrap();
let (prog, args) = SchrootDriver::wrap_command(
"printenv",
&["SOME_OPT".to_string()],
&[(
"SOME_OPT".to_string(),
"parallel=4 noch'eck \"x\"".to_string(),
)],
Some(dir.path().to_str().unwrap()),
);
let output = std::process::Command::new(&prog)
.arg(&args[0])
.arg(&args[1])
.output()
.unwrap();
assert!(output.status.success());
// printenv's output ends with a newline.
assert_eq!(
String::from_utf8_lossy(&output.stdout),
"parallel=4 noch'eck \"x\"\n"
);
}
}
+109
View File
@@ -0,0 +1,109 @@
//! POSIX-shell quoting for command strings assembled by the remote/chroot
//! execution contexts.
//!
//! Unlike [`super::local`] — which spawns programs directly through
//! `std::process::Command`, with no shell in between — the SSH, schroot and
//! unshare drivers ultimately hand a *string* to a shell (`ssh
//! channel.exec`, `sh -c`, `bash -c`). Every program name, argument,
//! path or environment value interpolated into such a string must be
//! quoted, or shell metacharacters (`;`, `|`, `&`, quotes, `$`, backticks,
//! globs, whitespace, ...) are reinterpreted by the shell: at best the
//! command breaks, at worst it executes injected input.
/// Quote `s` for safe interpolation into a POSIX shell command line.
///
/// The result is `s` wrapped in single quotes, with every embedded single
/// quote replaced by the standard `'\''` sequence (close the quoting, an
/// escaped literal quote, reopen). Whatever the input contains — spaces,
/// newlines, `"`, `'`, `$`, backticks, globs, `;` — the shell parses the
/// result back into exactly `s` as a single word. The empty string becomes
/// `''` (one empty argument, not zero arguments).
///
/// Use this for *every* value interpolated into a shell command string:
/// programs, arguments, `cd` targets, `env` assignments (both key and
/// value) and paths. It is safe (though redundant) to quote values that are
/// known to need no quoting.
pub(crate) fn shell_quote(s: &str) -> String {
let mut quoted = String::with_capacity(s.len() + 2);
quoted.push('\'');
for c in s.chars() {
if c == '\'' {
quoted.push_str("'\\''");
} else {
quoted.push(c);
}
}
quoted.push('\'');
quoted
}
#[cfg(test)]
mod tests {
use super::shell_quote;
#[test]
fn plain_word() {
assert_eq!(shell_quote("plain"), "'plain'");
}
#[test]
fn spaces_stay_one_word() {
assert_eq!(shell_quote("parallel=4 nocheck"), "'parallel=4 nocheck'");
assert_eq!(
shell_quote(" leading and trailing "),
"' leading and trailing '"
);
}
#[test]
fn embedded_single_quotes() {
assert_eq!(shell_quote("it's"), "'it'\\''s'");
assert_eq!(shell_quote("''"), r"''\'''\'''");
}
#[test]
fn double_quotes_and_metacharacters() {
assert_eq!(
shell_quote("say \"hi\" $HOME `id` ; | & * ?"),
"'say \"hi\" $HOME `id` ; | & * ?'"
);
}
#[test]
fn dollar_and_backtick_do_not_expand() {
assert_eq!(shell_quote("$HOME"), "'$HOME'");
assert_eq!(shell_quote("$(rm -rf /)"), "'$(rm -rf /)'");
assert_eq!(shell_quote("`touch /tmp/pwned`"), "'`touch /tmp/pwned`'");
}
#[test]
fn empty_string() {
assert_eq!(shell_quote(""), "''");
}
#[test]
fn unicode_preserved() {
assert_eq!(shell_quote("héllo→wörld ✓"), "'héllo→wörld ✓'");
}
#[test]
fn newlines_preserved() {
assert_eq!(shell_quote("a\nb"), "'a\nb'");
}
/// The definitive check: a real shell must parse the quoted string back
/// into the original value as a single argument, without expanding or
/// executing anything inside it.
#[cfg(unix)]
#[test]
fn round_trips_through_sh() {
let tricky = "a'b\"c $HOME `echo pwned` ; | & \n x*y";
let output = std::process::Command::new("sh")
.arg("-c")
.arg(format!("printf '%s' {}", shell_quote(tricky)))
.output()
.unwrap();
assert!(output.status.success());
assert_eq!(String::from_utf8_lossy(&output.stdout), tricky);
}
}
+135 -51
View File
@@ -2,6 +2,7 @@
/// Context driver: Copies over SFTP with ssh2, executes commands over ssh2 channels
use super::api::{ContextDriver, LineSink, Stream};
use super::capture::pump;
use super::shell::shell_quote;
use log::debug;
use ssh2;
use std::fs;
@@ -53,6 +54,41 @@ pub struct SshDriver {
pub port: Option<u16>,
}
impl SshDriver {
/// Build the remote shell command line: `export` assignments for `env`,
/// an optional `cd` to `cwd`, then `program` with its `args`.
///
/// The line is executed verbatim by the remote login shell through
/// `channel.exec`, so every component is POSIX-shell-quoted (see
/// [`shell_quote`]): metacharacters in arguments, paths or environment
/// values can neither break out of their word nor be expanded by the
/// remote shell.
fn build_command_line(
env: &[(String, String)],
cwd: Option<&str>,
program: &str,
args: &[String],
) -> String {
let mut cmd_line = String::new();
for (key, value) in env {
cmd_line.push_str(&format!(
"export {}={}; ",
shell_quote(key),
shell_quote(value)
));
}
if let Some(dir) = cwd {
cmd_line.push_str(&format!("cd {} && ", shell_quote(dir)));
}
cmd_line.push_str(&shell_quote(program));
for arg in args {
cmd_line.push(' ');
cmd_line.push_str(&shell_quote(arg));
}
cmd_line
}
}
impl ContextDriver for SshDriver {
fn ensure_available(&self, src: &Path, dest_root: &str) -> io::Result<PathBuf> {
let sess = connect_ssh(&self.host, self.user.as_deref(), self.port)?;
@@ -106,22 +142,7 @@ impl ContextDriver for SshDriver {
// Construct command line with env vars
// TODO: No, use ssh2 channel.set_env
let mut cmd_line = String::new();
for (key, value) in env {
cmd_line.push_str(&format!(
"export {}='{}'; ",
key,
value.replace("'", "'\\''")
));
}
if let Some(dir) = cwd {
cmd_line.push_str(&format!("cd {} && ", dir));
}
cmd_line.push_str(program);
for arg in args {
cmd_line.push(' ');
cmd_line.push_str(arg); // TODO: escape
}
let cmd_line = Self::build_command_line(env, cwd, program, args);
debug!("Executing SSH command: {}", cmd_line);
@@ -152,23 +173,8 @@ impl ContextDriver for SshDriver {
let sess = connect_ssh(&self.host, self.user.as_deref(), self.port)?;
let mut channel = sess.channel_session().map_err(io::Error::other)?;
// Construct command line with env vars (same escaping as `run`)
let mut cmd_line = String::new();
for (key, value) in env {
cmd_line.push_str(&format!(
"export {}='{}'; ",
key,
value.replace("'", "'\\''")
));
}
if let Some(dir) = cwd {
cmd_line.push_str(&format!("cd {} && ", dir));
}
cmd_line.push_str(program);
for arg in args {
cmd_line.push(' ');
cmd_line.push_str(arg); // TODO: escape
}
// Construct command line with env vars (same quoting as `run`)
let cmd_line = Self::build_command_line(env, cwd, program, args);
debug!("Executing SSH command (captured): {}", cmd_line);
@@ -200,23 +206,8 @@ impl ContextDriver for SshDriver {
let sess = connect_ssh(&self.host, self.user.as_deref(), self.port)?;
let mut channel = sess.channel_session().map_err(io::Error::other)?;
// Construct command line with env vars
let mut cmd_line = String::new();
for (key, value) in env {
cmd_line.push_str(&format!(
"export {}='{}'; ",
key,
value.replace("'", "'\\''")
));
}
if let Some(dir) = cwd {
cmd_line.push_str(&format!("cd {} && ", dir));
}
cmd_line.push_str(program);
for arg in args {
cmd_line.push(' ');
cmd_line.push_str(arg); // TODO: escape
}
// Construct command line with env vars (same quoting as `run`)
let cmd_line = Self::build_command_line(env, cwd, program, args);
channel.exec(&cmd_line).map_err(io::Error::other)?;
@@ -266,7 +257,11 @@ impl ContextDriver for SshDriver {
let sess = connect_ssh(&self.host, self.user.as_deref(), self.port)?;
let mut channel = sess.channel_session().map_err(io::Error::other)?;
// TODO: use sftp
let cmd = format!("cp -a {:?} {:?}", src, dest);
let cmd = format!(
"cp -a {} {}",
shell_quote(&src.to_string_lossy()),
shell_quote(&dest.to_string_lossy())
);
debug!("Executing remote copy: {}", cmd);
channel.exec(&cmd).map_err(io::Error::other)?;
channel.wait_close().map_err(io::Error::other)?;
@@ -293,6 +288,13 @@ impl ContextDriver for SshDriver {
}
let mut remote_file = sftp.create(path).map_err(io::Error::other)?;
remote_file.write_all(content.as_bytes())?;
// Close explicitly: the `Drop` impl of `ssh2::File` discards a
// close-time error ("too late to recover"), silently truncating the
// remote file. Writes are unbuffered (`Write::flush` is a no-op), so
// no flush is needed before closing.
remote_file.close().map_err(|e| {
io::Error::other(format!("Failed to close remote file {:?}: {}", path, e))
})?;
Ok(())
}
@@ -304,6 +306,14 @@ impl ContextDriver for SshDriver {
Err(_) => Ok(false),
}
}
fn is_dir(&self, path: &Path) -> io::Result<bool> {
let sess = connect_ssh(&self.host, self.user.as_deref(), self.port)?;
let sftp = sess.sftp().map_err(io::Error::other)?;
// Same error tolerance as `exists`: an unreachable path is not a
// directory, and the caller decides what absence means.
Ok(sftp.stat(path).map(|stat| stat.is_dir()).unwrap_or(false))
}
}
impl SshDriver {
@@ -332,6 +342,18 @@ impl SshDriver {
io::Error::other(format!("Failed to create remote file {:?}: {}", dest, e))
})?;
io::copy(&mut file, &mut remote_file)?;
// Close explicitly: quota-exceeded and similar failures only
// surface in the final ACKs and the close handshake, and the
// `Drop` impl of `ssh2::File` discards that error ("too late to
// recover"), leaving a truncated remote file behind. Writes are
// unbuffered (`ssh2::File`'s `Write::flush` is a no-op), so no
// flush is needed before closing.
remote_file.close().map_err(|e| {
io::Error::other(format!(
"Failed to close remote file {:?} after upload: {}",
dest, e
))
})?;
}
Ok(())
}
@@ -360,3 +382,65 @@ impl SshDriver {
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::SshDriver;
/// Program, arguments and the `cd` target must each be a single,
/// quoted word; `$` in the cwd must not be expanded.
#[test]
fn command_line_quotes_program_args_and_cwd() {
let line = SshDriver::build_command_line(
&[],
Some("/tmp/some dir/$HOST"),
"make",
&["install".to_string(), "PREFIX=/opt/my app".to_string()],
);
assert_eq!(
line,
"cd '/tmp/some dir/$HOST' && 'make' 'install' 'PREFIX=/opt/my app'"
);
}
/// Env keys and values are quoted too (values used to be escaped by
/// hand, keys and everything else not at all).
#[test]
fn command_line_quotes_env_keys_and_values() {
let line = SshDriver::build_command_line(
&[(
"DEB_BUILD_OPTIONS".to_string(),
"parallel=4 nocheck".to_string(),
)],
None,
"dpkg-buildpackage",
&["-us".to_string(), "-uc".to_string()],
);
assert_eq!(
line,
"export 'DEB_BUILD_OPTIONS'='parallel=4 nocheck'; 'dpkg-buildpackage' '-us' '-uc'"
);
}
/// The definitive check: a real shell must execute the assembled line
/// exactly as intended — one argument through, one env value verbatim —
/// even when both contain quotes, spaces and `$`.
#[cfg(unix)]
#[test]
fn command_line_survives_shell_parsing() {
let dir = tempfile::tempdir().unwrap();
let line = SshDriver::build_command_line(
&[("OPT".to_string(), "a b'c \"$d\"".to_string())],
Some(dir.path().to_str().unwrap()),
"printenv",
&["OPT".to_string()],
);
let output = std::process::Command::new("sh")
.arg("-c")
.arg(&line)
.output()
.unwrap();
assert!(output.status.success());
assert_eq!(String::from_utf8_lossy(&output.stdout), "a b'c \"$d\"\n");
}
}
+98 -24
View File
@@ -1,4 +1,5 @@
use super::api::{Context, ContextCommand, ContextDriver, LineSink};
use super::shell::shell_quote;
use log::debug;
use std::fs;
use std::io;
@@ -295,36 +296,40 @@ impl ContextDriver for UnshareDriver {
fn create_temp_dir(&self) -> io::Result<String> {
// Create a temporary directory inside the chroot with unique naming
// Sub-second precision and an atomic create, like the local
// driver: concurrent callers racing on the same name must not
// share a directory, so an existing target falls through to the
// next attempt instead of a probe-then-create window.
let base_timestamp = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs();
.as_millis();
let mut attempt = 0;
loop {
let work_dir_name = if attempt == 0 {
format!("pkh-build-{}", base_timestamp)
format!("pkh-build-{base_timestamp}")
} else {
format!("pkh-build-{}-{}", base_timestamp, attempt)
format!("pkh-build-{base_timestamp}-{attempt}")
};
let work_dir_inside_chroot = format!("/tmp/{}", work_dir_name);
let work_dir_inside_chroot = format!("/tmp/{work_dir_name}");
let host_path = Path::new(&self.path).join("tmp").join(&work_dir_name);
// Check if directory already exists
if host_path.exists() {
attempt += 1;
continue;
}
// Create the directory on the host filesystem
std::fs::create_dir_all(&host_path)?;
match std::fs::create_dir(&host_path) {
Ok(()) => {
debug!(
"Created work directory: {} (host: {})",
work_dir_inside_chroot,
host_path.display()
);
}
Err(e) if e.kind() == io::ErrorKind::AlreadyExists => {
attempt += 1;
continue;
}
Err(e) => return Err(e),
}
// Return the path as it appears inside the chroot
return Ok(work_dir_inside_chroot);
@@ -351,6 +356,11 @@ impl ContextDriver for UnshareDriver {
let host_path = Path::new(&self.path).join(path.to_string_lossy().trim_start_matches('/'));
self.parent().exists(&host_path)
}
fn is_dir(&self, path: &Path) -> io::Result<bool> {
let host_path = Path::new(&self.path).join(path.to_string_lossy().trim_start_matches('/'));
Ok(host_path.is_dir())
}
}
impl UnshareDriver {
@@ -484,21 +494,85 @@ impl UnshareDriver {
// Build the bash command: set up /dev/pts and run the program
// /proc should already be bind-mounted from the host before entering the namespace
let program_args = args
.iter()
.map(|a| format!("\"{a}\""))
.collect::<Vec<_>>()
.join(" ");
cmd.arg("--")
.arg("bash")
.arg("-c")
.arg(format!(
"mkdir -p /dev/pts; mount -t devpts devpts /dev/pts 2>/dev/null || true; touch /dev/ptmx; mount --bind /dev/pts/ptmx /dev/ptmx 2>/dev/null || true; {} {}",
program,
program_args
));
.arg(build_namespace_script(program, args));
cmd
}
}
/// Build the shell script executed by `bash -c` inside the user namespace:
/// bring up `/dev/pts`, then run `program` with `args`.
///
/// The script is parsed by bash, so the program and every argument are
/// POSIX-shell-quoted (see [`shell_quote`]): quotes, `$`, backticks or
/// whitespace inside them can neither split the command into different
/// words nor trigger expansion. (Previously arguments were wrapped in
/// unescaped double quotes, so a `"` in an argument broke out and
/// `$`/backticks still expanded.)
fn build_namespace_script(program: &str, args: &[String]) -> String {
let mut script = String::from(
"mkdir -p /dev/pts; mount -t devpts devpts /dev/pts 2>/dev/null || true; touch /dev/ptmx; mount --bind /dev/pts/ptmx /dev/ptmx 2>/dev/null || true; ",
);
script.push_str(&shell_quote(program));
for arg in args {
script.push(' ');
script.push_str(&shell_quote(arg));
}
script
}
#[cfg(test)]
mod tests {
use super::build_namespace_script;
fn tail_after_devpts_setup(script: &str) -> &str {
script
.rsplit_once("|| true; ")
.map(|(_, rest)| rest)
.unwrap()
.trim_end()
}
/// Program and arguments must each be a single, quoted word at the end
/// of the `/dev/pts` setup script.
#[test]
fn script_quotes_program_and_args() {
let script = build_namespace_script(
"make",
&[
"install".to_string(),
"a b".to_string(),
"PREFIX=/opt/my app".to_string(),
],
);
assert_eq!(
tail_after_devpts_setup(&script),
"'make' 'install' 'a b' 'PREFIX=/opt/my app'"
);
}
/// An argument containing a double quote must not break out of the
/// script (arguments used to be wrapped in unescaped `"`), and `$`/
/// backticks must stay literal for bash.
#[test]
fn script_neutralizes_quotes_and_expansions() {
let script = build_namespace_script(
"echo",
&["$(touch /tmp/pwned) `id` \"; rm -rf /\"".to_string()],
);
assert_eq!(
tail_after_devpts_setup(&script),
"'echo' '$(touch /tmp/pwned) `id` \"; rm -rf /\"'"
);
}
/// Empty arguments must survive as one empty word (`''`), not vanish.
#[test]
fn script_preserves_empty_args() {
let script = build_namespace_script("prog", &[String::new(), "x".to_string()]);
assert_eq!(tail_after_devpts_setup(&script), "'prog' '' 'x'");
}
}
+72
View File
@@ -0,0 +1,72 @@
//! Embedding convention for the static reference data files (`data/*.yml`)
//!
//! Reference data that changes independently of the code — distro series
//! pointers, pinned SSH host keys, package quirks — lives in YAML files
//! under `data/` at the repo root instead of hardcoded in the source, so
//! it is updatable in one reviewable place.
//!
//! This module is deliberately not a central registry: each file is
//! embedded by the module that owns it (distro_info.rs owns
//! data/distro_info.yml, launchpad.rs owns data/launchpad.yml,
//! apt/keyring.rs owns data/keyserver.yml, new/origin.rs owns
//! data/forges.yml, put/ssh.rs owns data/host_keys.yml, quirks.rs owns
//! data/quirks.yml) through the [`embed_data!`] macro below, so data
//! and its accessors stay together and a diff touching one domain cannot
//! half-touch another. The macro embeds the file at compile time and
//! parses it once into a `lazy_static` on first use; since the data ships
//! inside the binary, a parse failure is a build-time bug that cannot be
//! recovered from at runtime, and the macro panics on it.
//!
//! Paths and URLs in the data files carry their variable parts as `{name}`
//! placeholders, substituted with `str::replace` at the use site — no
//! template engine.
/// Embed one YAML data file as a lazily-parsed static, following the
/// convention documented at the module level.
///
/// Takes the visibility of the generated static (none for private, `pub` or
/// `pub(crate)`-style), its name, its struct type (which stays defined in
/// the owning module, next to its accessors) and the file path relative to
/// the invoking source file (`"../data/distro_info.yml"` from
/// `src/distro_info.rs`, `"../../data/host_keys.yml"` from
/// `src/put/ssh.rs`, ...), and expands to the house `include_str!` →
/// `lazy_static` → parse pattern — only the embed+parse boilerplate is
/// generated.
///
/// ```ignore
/// embed_data! {
/// static ref MY_DATA: MyData = "../data/my_data.yml"
/// }
/// ```
macro_rules! embed_data {
// Internal arm: the visibility arrives wrapped in parentheses (empty for
// private statics) because `lazy_static!` only re-matches literal
// `pub`/`pub(...)` token sequences, not an opaque forwarded `vis`.
(@expand ($($vis:tt)*) static ref $name:ident : $ty:ty = $path:literal) => {
lazy_static::lazy_static! {
// The YAML is include_str!'d at compile time and statically
// valid; if it ever failed to parse it would be a build-time bug
// that cannot be recovered from at runtime, so panicking here is
// acceptable.
$($vis)* static ref $name: $ty = serde_yaml::from_str(include_str!($path))
.expect(concat!(
"built-in ",
$path,
" data is statically valid and must parse"
));
}
};
(static ref $name:ident : $ty:ty = $path:literal) => {
$crate::data::embed_data!(@expand () static ref $name : $ty = $path);
};
(pub static ref $name:ident : $ty:ty = $path:literal) => {
$crate::data::embed_data!(@expand (pub) static ref $name : $ty = $path);
};
(pub ($($vis:tt)+) static ref $name:ident : $ty:ty = $path:literal) => {
$crate::data::embed_data!(@expand (pub ($($vis)+)) static ref $name : $ty = $path);
};
}
/// Makes the macro available through the module path
/// (`use crate::data::embed_data;`)
pub(crate) use embed_data;
+261 -101
View File
@@ -1,24 +1,19 @@
use crate::context::Context;
use log::debug;
use std::collections::HashMap;
use std::error::Error;
use std::sync::Arc;
/// Set environment variables for cross-compilation
pub fn setup_environment(
env: &mut HashMap<String, String>,
arch: &str,
ctx: Arc<Context>,
) -> Result<(), Box<dyn Error>> {
let dpkg_architecture = String::from_utf8(
ctx.command("dpkg-architecture")
.arg("-a")
.arg(arch)
.output()?
.stdout,
)?;
/// Parse 'dpkg-architecture' output (KEY=value lines) into a set of
/// environment variables. Unexpected lines (e.g. warnings on stderr leaking
/// into stdout) are skipped instead of causing a failure.
fn parse_dpkg_architecture_output(output: &str, env: &mut HashMap<String, String>) {
let env_var_regex = regex::Regex::new(r"(?<key>.*)=(?<value>.*)").unwrap();
for l in dpkg_architecture.lines() {
let capture = env_var_regex.captures(l).unwrap();
for l in output.lines() {
let Some(capture) = env_var_regex.captures(l) else {
debug!("Skipping unexpected dpkg-architecture output line: '{l}'");
continue;
};
let key = capture.name("key").unwrap().as_str().to_string();
let value = capture.name("value").unwrap().as_str().to_string();
@@ -28,13 +23,99 @@ pub fn setup_environment(
env.insert("CROSS_COMPILE".to_string(), format!("{value}-"));
}
}
}
/// Set environment variables for cross-compilation
pub fn setup_environment(
env: &mut HashMap<String, String>,
arch: &str,
ctx: Arc<Context>,
) -> Result<(), Box<dyn Error>> {
let output = ctx
.command("dpkg-architecture")
.arg("-a")
.arg(arch)
.output()
.map_err(|e| {
format!(
"Failed to run 'dpkg-architecture -a {arch}': {e}. \
Is 'dpkg-dev' installed?"
)
})?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(format!(
"'dpkg-architecture -a {}' failed with status: {}.{}",
arch,
output.status,
if stderr.trim().is_empty() {
String::new()
} else {
format!("\ndpkg-architecture output:\n{}", stderr.trim())
}
)
.into());
}
let dpkg_architecture = String::from_utf8(output.stdout)
.map_err(|e| format!("Invalid UTF-8 in dpkg-architecture output: {e}"))?;
parse_dpkg_architecture_output(&dpkg_architecture, env);
// In-tree tools locate their libraries with the *host* pkg-config during
// cross builds (the kernel's tools/build feature checks derive their
// cflags/ldflags from `pkg-config --cflags/--libs`), whose search path
// only covers the build architecture's pkgconfig dirs. Point it at the
// target's so `libtraceevent` & co resolve to target-arch libraries:
// linux-riscv cross builds die in rtla's Makefile.config otherwise, even
// with the target -dev packages installed.
if let Some(multiarch) = env.get("DEB_HOST_MULTIARCH").cloned() {
env.insert(
"PKG_CONFIG_LIBDIR".to_string(),
format!("/usr/lib/{multiarch}/pkgconfig:/usr/share/pkgconfig"),
);
}
env.insert("DEB_BUILD_PROFILES".to_string(), "cross".to_string());
Ok(())
}
/// The suites a cross-build environment enables for `series`: the series
/// itself plus the distro data's cross pockets (`<series>-updates`,
/// `<series>-backports`, `<series>-security`), plus the explicitly
/// requested `pocket` when one is given ('proposed' stays opt-in exactly
/// this way — it is not a cross pocket). Shared by the source-adjusting
/// pass and the added mirror entry, which used to duplicate the list.
fn cross_suites(
series: &str,
pocket: Option<&str>,
dist: &str,
) -> Result<Vec<String>, Box<dyn Error>> {
let mut suites = vec![series.to_string()];
for p in crate::distro_info::get_cross_pockets(dist)? {
suites.push(format!("{series}-{p}"));
}
if let Some(p) = pocket {
let pocket_suite = format!("{series}-{p}");
if !suites.contains(&pocket_suite) {
suites.push(pocket_suite);
}
}
Ok(suites)
}
/// Ensure that repositories for target architecture are available
/// This also handles the 'ports.ubuntu.com' vs 'archive.ubuntu.com' on Ubuntu
///
/// On Ubuntu hosts, driven by the bundled distro data
/// (`data/distro_info.yml`): the official sources served by the mirror of
/// the local architecture (the primary archive and its security sibling)
/// are scoped to it and carry every component and cross-build suite, and
/// the mirror serving the target architecture (ports, for the non-local
/// ones) is added when no existing source serves the arch from it yet.
/// Debian hosts are left alone: one mirror serves every architecture, so
/// the host's own sources already cover the target — the os-release gate
/// below is what makes that a data conclusion instead of hardcoding.
pub fn ensure_repositories(
arch: &str,
series: &str,
@@ -44,124 +125,203 @@ pub fn ensure_repositories(
let local_arch = crate::get_current_arch();
// Add target ('host') architecture
ctx.command("dpkg")
let status = ctx
.command("dpkg")
.arg("--add-architecture")
.arg(arch)
.status()?;
.status()
.map_err(|e| format!("Failed to run 'dpkg --add-architecture {arch}': {e}"))?;
if !status.success() {
return Err(format!(
"'dpkg --add-architecture {}' failed with status: {}",
arch, status
)
.into());
}
// Check if we are on Ubuntu
let os_release = String::from_utf8(ctx.command("cat").arg("/etc/os-release").output()?.stdout)?;
if !os_release.contains("ID=ubuntu") {
return Ok(());
}
let dist = "ubuntu";
// Load existing sources
let mut sources = crate::apt::sources::load(Some(ctx.clone()))?;
// The mirrors serving each side of the cross build (primary for the
// local architectures, ports for the others) and the distro data's
// components and suites
let local_mirror = crate::distro_info::mirror_for_arch(dist, &local_arch)?;
let target_mirror = crate::distro_info::mirror_for_arch(dist, arch)?;
let components = crate::distro_info::get_dist_components(dist)?;
let required_suites = cross_suites(series, pocket, dist)?;
// Ensure all components are enabled for the primary architecture
for source in &mut sources {
if source.uri.contains("archive.ubuntu.com") || source.uri.contains("security.ubuntu.com") {
// Official sources served by the local mirror (the primary archive
// and its security sibling); ports serves the other architectures
// and is configured below instead
if !crate::distro_info::is_mirror_source(local_mirror, &source.uri) {
continue;
}
// Scope to local_arch if not already scoped
if source.architectures.is_empty() {
source.architectures.push(local_arch.clone());
}
// Ensure all components are present
let required_components = ["main", "restricted", "universe", "multiverse"];
for &comp in &required_components {
if !source.components.contains(&comp.to_string()) {
source.components.push(comp.to_string());
for comp in &components {
if !source.components.contains(comp) {
source.components.push(comp.clone());
}
}
// Ensure all suites (pockets) are enabled, excluding 'proposed'
// unless explicitly requested through the 'pocket' option
let mut required_suites = vec![
series.to_string(),
format!("{}-updates", series),
format!("{}-backports", series),
format!("{}-security", series),
];
if let Some(p) = pocket {
let pocket_suite = format!("{series}-{p}");
if !required_suites.contains(&pocket_suite) {
required_suites.push(pocket_suite);
}
}
for suite in required_suites {
if !source.suite.contains(&suite) {
source.suite.push(suite);
}
// Ensure all suites (pockets) are enabled
for suite in &required_suites {
if !source.suite.contains(suite) {
source.suite.push(suite.clone());
}
}
}
// Check if ports repository already exists for the target architecture
let has_ports = sources
.iter()
.any(|s| s.uri.contains("ports.ubuntu.com") && s.architectures.contains(&arch.to_string()));
// Check whether an existing source already serves the target
// architecture from its mirror (e.g. the ports mirror for a
// non-local arch); when cross-building for the local architecture,
// the primary sources above already do
let has_target = sources.iter().any(|s| {
crate::distro_info::is_mirror_source(target_mirror, &s.uri)
&& s.architectures.contains(&arch.to_string())
});
if !has_ports {
// Add ports repository for the target architecture
let mut ports_suites = vec![
series.to_string(),
format!("{series}-updates"),
format!("{series}-backports"),
format!("{series}-security"),
];
if let Some(p) = pocket {
let pocket_suite = format!("{series}-{p}");
if !ports_suites.contains(&pocket_suite) {
ports_suites.push(pocket_suite);
}
}
let ports_entry = crate::apt::sources::SourceEntry {
if !has_target {
// Add the target architecture's mirror (ports for the non-local
// architectures on Ubuntu)
let mirror_entry = crate::apt::sources::SourceEntry {
enabled: true,
components: vec![
"main".to_string(),
"restricted".to_string(),
"universe".to_string(),
"multiverse".to_string(),
],
kind: crate::apt::sources::SourceKind::Deb,
components: components.clone(),
architectures: vec![arch.to_string()],
uri: "http://ports.ubuntu.com/ubuntu-ports".to_string(),
suite: ports_suites,
uri: target_mirror.url.clone(),
signed_by: None,
trusted: None,
suite: required_suites.clone(),
// No origin: saved to the pkh-owned added-sources file
origin: None,
};
sources.push(ports_entry);
sources.push(mirror_entry);
}
// Save the updated sources
// Try to save in DEB822 format first, fall back to legacy format
let deb822_path = "/etc/apt/sources.list.d/ubuntu.sources";
if ctx
.command("test")
.arg("-f")
.arg(deb822_path)
.status()?
.success()
{
// For DEB822 format, we need to reconstruct the file content
let mut content = String::new();
for source in &sources {
if !source.enabled {
continue;
}
content.push_str("Types: deb\n");
content.push_str(&format!("URIs: {}\n", source.uri));
content.push_str(&format!("Suites: {}\n", source.suite.join(" ")));
content.push_str(&format!("Components: {}\n", source.components.join(" ")));
content.push_str("Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg\n");
content.push_str(&format!(
"Architectures: {}\n",
source.architectures.join(" ")
));
content.push('\n');
}
ctx.write_file(std::path::Path::new(deb822_path), &content)?;
} else {
// Fall back to legacy format
crate::apt::sources::save_legacy(Some(ctx.clone()), sources, "/etc/apt/sources.list")?;
}
// Save the updated sources: each entry is written back to its origin
// file in its own format (keeping its own Signed-By and Enabled state),
// and the new mirror entry goes to the pkh-owned added-sources file
crate::apt::sources::save(Some(ctx.clone()), sources)?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_dpkg_architecture_output() {
let output = "DEB_BUILD_ARCH=amd64\n\
DEB_HOST_ARCH=arm64\n\
DEB_HOST_GNU_TYPE=aarch64-linux-gnu\n";
let mut env = HashMap::new();
parse_dpkg_architecture_output(output, &mut env);
assert_eq!(env.get("DEB_BUILD_ARCH").map(String::as_str), Some("amd64"));
assert_eq!(env.get("DEB_HOST_ARCH").map(String::as_str), Some("arm64"));
assert_eq!(
env.get("DEB_HOST_GNU_TYPE").map(String::as_str),
Some("aarch64-linux-gnu")
);
// Derived variable for the GNU type
assert_eq!(
env.get("CROSS_COMPILE").map(String::as_str),
Some("aarch64-linux-gnu-")
);
}
#[test]
fn test_parse_dpkg_architecture_output_skips_unexpected_lines() {
// Unexpected lines (warnings on stdout, empty lines) must be skipped
// instead of panicking
let output = "dpkg-architecture: warning: something odd happened\n\
\n\
DEB_HOST_GNU_TYPE=arm-linux-gnueabihf\n\
not an environment variable assignment\n";
let mut env = HashMap::new();
parse_dpkg_architecture_output(output, &mut env);
assert_eq!(
env.get("DEB_HOST_GNU_TYPE").map(String::as_str),
Some("arm-linux-gnueabihf")
);
assert_eq!(
env.get("CROSS_COMPILE").map(String::as_str),
Some("arm-linux-gnueabihf-")
);
assert_eq!(env.len(), 2);
}
/// The suite list a cross-build environment enables: the series, its
/// cross pockets from the distro data (updates, backports, security),
/// and the explicitly requested pocket — which is the only way
/// 'proposed' gets in.
#[test]
fn test_cross_suites_from_distro_data() {
assert_eq!(
cross_suites("noble", None, "ubuntu").unwrap(),
vec![
"noble".to_string(),
"noble-updates".to_string(),
"noble-backports".to_string(),
"noble-security".to_string()
]
);
// An explicitly requested pocket is added (not duplicated when it
// is already a cross pocket).
assert_eq!(
cross_suites("noble", Some("proposed"), "ubuntu").unwrap(),
vec![
"noble".to_string(),
"noble-updates".to_string(),
"noble-backports".to_string(),
"noble-security".to_string(),
"noble-proposed".to_string()
]
);
assert_eq!(
cross_suites("noble", Some("updates"), "ubuntu").unwrap(),
cross_suites("noble", None, "ubuntu").unwrap()
);
assert!(cross_suites("noble", None, "not-a-distro").is_err());
}
/// setup_environment exports the target multiarch pkg-config libdir:
/// tools' feature checks run the *host* pkg-config, which must find the
/// target's .pc files (rtla hard-errors on libtraceevent otherwise,
/// failing linux-riscv cross builds despite the target -dev packages
/// being installed).
#[test]
fn test_setup_environment_exports_cross_pkg_config_libdir() {
let mut env = HashMap::new();
let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local).unwrap());
setup_environment(&mut env, "riscv64", ctx).unwrap();
assert_eq!(
env.get("PKG_CONFIG_LIBDIR").map(String::as_str),
Some("/usr/lib/riscv64-linux-gnu/pkgconfig:/usr/share/pkgconfig")
);
assert_eq!(
env.get("DEB_BUILD_PROFILES").map(String::as_str),
Some("cross")
);
}
}
+376 -44
View File
@@ -1,20 +1,206 @@
use crate::context::{self, Context, ContextConfig};
use crate::ui::deb::{DebUi, Phase};
use crate::deb::{Phase, enter_phase};
use crate::interrupt::CleanupHookGuard;
use crate::report::BuildView;
use directories::ProjectDirs;
use std::error::Error;
use std::fs;
use std::path::{Path, PathBuf};
use std::process::Command;
use std::sync::Arc;
use tar::Archive;
use xz2::read::XzDecoder;
// ---------------------------------------------------------------------------
// Interrupt-time chroot cleanup
//
// On Ctrl-C, the watchdog in `crate::interrupt` runs the hook registered in
// [`EphemeralContextGuard::new_with_context`] right before exiting — the
// interrupt sequence skips all destructors, which would otherwise leak the
// freshly bootstrapped chroot under /tmp together with its bind-mounted
// /proc and any overlayfs mounts.
// ---------------------------------------------------------------------------
/// Interrupt-time cleanup of an ephemeral chroot: unmount every host-side
/// mount at or below `chroot_path` (the /proc bind mount, any overlay mounts)
/// and then remove the directory tree.
///
/// Unlike [`EphemeralContextGuard::drop`], this deliberately does NOT go
/// through the context manager, the ephemeral context's driver (whose
/// `cleanup()` unmounts the tracked overlays) or the base context's command
/// builder: interrupt-time hooks must be self-contained, and those
/// machineries may be mid-mutation on the interrupted thread. Instead it
/// only reads /proc/mounts and spawns umount/rm directly.
///
/// It also differs from `drop` in that it removes the chroot regardless of
/// the build result: the build was aborted, and leaving a still-mounted
/// chroot behind is exactly the leak this hook exists to prevent.
///
/// Best-effort by design: if a child process still holds a mount busy or
/// privilege escalation is unavailable, individual steps fail; failures are
/// logged (pointing at `pkh prune` for the leftovers) and never panic.
fn sigint_cleanup_chroot(chroot_path: &Path) {
let is_root = unsafe { libc::geteuid() } == 0;
// Unmount children before parents: /proc/mounts lists mounts roughly in
// creation order, so walk it in reverse
let mounts = host_mounts_under(chroot_path);
for mount_point in mounts.into_iter().rev() {
if unmount_path(&mount_point, is_root) {
log::debug!(
"Unmounted {} during interrupt cleanup",
mount_point.display()
);
} else {
log::error!(
"Failed to unmount {} during interrupt cleanup; \
run `pkh prune` once the mount is free",
mount_point.display()
);
}
}
// Remove the chroot tree itself (tolerates a missing directory). A
// child the Ctrl+C interrupted may still be finishing its writeout —
// dpkg defers SIGINT until it reaches a safe state — so retry while rm
// reports the tree non-empty instead of leaving it half-removed.
const RETRIES: usize = 10;
const RETRY_DELAY: std::time::Duration = std::time::Duration::from_millis(300);
let mut last = None;
for attempt in 0..=RETRIES {
if attempt > 0 {
std::thread::sleep(RETRY_DELAY);
}
last = Some(
privileged_command("rm", is_root)
.arg("-rf")
.arg(chroot_path)
.status(),
);
if matches!(&last, Some(Ok(status)) if status.success()) {
break;
}
}
match last {
Some(Ok(status)) if status.success() => {
log::debug!(
"Removed chroot {} during interrupt cleanup",
chroot_path.display()
);
}
Some(Ok(status)) => {
log::error!(
"Failed to remove chroot {} during interrupt cleanup \
(rm exited with {status}); run `pkh prune`",
chroot_path.display()
);
}
Some(Err(e)) => {
log::error!(
"Failed to run rm for chroot {} during interrupt cleanup: {e}; run `pkh prune`",
chroot_path.display()
);
}
None => unreachable!("at least one rm attempt ran"),
}
}
/// Build a `Command` for `program`, wrapped in non-interactive sudo when not
/// running as root: interrupt cleanup must never block on a password prompt,
/// so without cached credentials the command fails fast and is logged instead
fn privileged_command(program: &str, is_root: bool) -> Command {
if is_root {
Command::new(program)
} else {
let mut cmd = Command::new("sudo");
cmd.arg("-n").arg(program);
cmd
}
}
/// Unmount `path`, falling back to a lazy unmount if the first attempt fails
/// because something still holds the mount busy (e.g. an interrupted child
/// that has not exited yet); returns whether the mount is gone
fn unmount_path(path: &Path, is_root: bool) -> bool {
if privileged_command("umount", is_root)
.arg(path)
.status()
.is_ok_and(|s| s.success())
{
return true;
}
privileged_command("umount", is_root)
.arg("-l")
.arg(path)
.status()
.is_ok_and(|s| s.success())
}
/// Collect the host-side mount points at or below `base`, in /proc/mounts
/// order (empty if /proc/mounts cannot be read)
fn host_mounts_under(base: &Path) -> Vec<PathBuf> {
let mut mounts = Vec::new();
let Ok(mounts_text) = fs::read_to_string("/proc/mounts") else {
return mounts;
};
// Compare against the canonical path: /proc/mounts shows resolved paths,
// while the chroot path may go through a symlinked TMPDIR
let base = base.canonicalize().unwrap_or_else(|_| base.to_path_buf());
for line in mounts_text.lines() {
let mut fields = line.split_whitespace();
let (Some(_device), Some(mount_point)) = (fields.next(), fields.next()) else {
continue;
};
let path = PathBuf::from(unescape_mount_field(mount_point));
if path.starts_with(&base) && !mounts.contains(&path) {
mounts.push(path);
}
}
mounts
}
/// Decode the octal escapes /proc/mounts uses in its path fields
/// (`\040` for space, `\011` for tab, `\012` for newline, `\134` for backslash)
fn unescape_mount_field(field: &str) -> String {
let bytes = field.as_bytes();
let mut out = Vec::with_capacity(bytes.len());
let mut i = 0;
while i < bytes.len() {
if bytes[i] == b'\\'
&& i + 4 <= bytes.len()
&& bytes[i + 1..i + 4]
.iter()
.all(|b| (b'0'..=b'7').contains(b))
&& let Ok(value) = u8::from_str_radix(&field[i + 1..i + 4], 8)
{
out.push(value);
i += 4;
} else {
out.push(bytes[i]);
i += 1;
}
}
String::from_utf8_lossy(&out).into_owned()
}
/// An ephemeral unshare context guard that creates and manages a temporary chroot environment
/// for building packages with unshare permissions.
pub struct EphemeralContextGuard {
previous_context: String,
/// The ephemeral build context this guard created (an unshare context
/// bound to the chroot, parented on the base context). Held explicitly so
/// cleanup and the build itself never depend on the process-global
/// "current" context, which concurrent builds swap for their own.
ephemeral_ctx: Arc<Context>,
/// The context that was current (globally) when this guard was created,
/// restored on drop. Saving the handle instead of a config name is what
/// keeps concurrent builds from restoring over each other.
previous_context: Arc<Context>,
chroot_path: PathBuf,
build_succeeded: bool,
base_ctx: Arc<Context>,
/// Registration of the interrupt-time cleanup hook; deregistered when
/// this guard drops, so the hook can never fire after the normal cleanup
cleanup_hook: Option<CleanupHookGuard>,
}
impl EphemeralContextGuard {
@@ -29,9 +215,13 @@ impl EphemeralContextGuard {
series: &str,
arch: Option<&str>,
base_ctx: Arc<Context>,
ui: Option<Arc<DebUi>>,
view: &dyn BuildView,
) -> Result<Self, Box<dyn Error>> {
let current_context_name = context::manager().current_name();
// Save the globally-installed context so Drop can restore exactly
// this handle: concurrent builds install their own ephemeral
// overrides, so the only safe restoration value is the one observed
// before this guard swapped anything in.
let previous_context = context::current();
// Create a temporary directory for the chroot
let chroot_path_str = base_ctx.create_temp_dir()?;
@@ -44,30 +234,91 @@ impl EphemeralContextGuard {
chroot_path.display()
);
// Download and extract the chroot tarball
Self::download_and_extract_chroot(series, arch, &chroot_path, base_ctx.clone(), &ui)
.await?;
// Register the interrupt-time cleanup hook before any heavy work: if
// the user hits Ctrl-C during bootstrap or the build itself, the
// interrupt watchdog unmounts and removes the chroot through this
// hook (see `sigint_cleanup_chroot`). This only works for a local
// base context: the hook must be self-contained (stored path +
// direct umount/rm subprocesses) and cannot go through `base_ctx`.
// For remote or nested bases the chroot lives elsewhere, and
// leftovers stay handled by `pkh prune` as before.
let cleanup_hook = if matches!(base_ctx.config, ContextConfig::Local) {
Some(crate::interrupt::register_cleanup_hook(Box::new({
let chroot_path = chroot_path.clone();
move || sigint_cleanup_chroot(&chroot_path)
})))
} else {
log::debug!(
"Base context is not local; skipping interrupt-time cleanup registration for {}",
chroot_path.display()
);
None
};
// Switch to an ephemeral context to build the package in the chroot
context::manager().set_current_ephemeral(Context::new(ContextConfig::Unshare {
// Download and extract the chroot tarball
if let Err(e) =
Self::download_and_extract_chroot(series, arch, &chroot_path, base_ctx.clone(), view)
.await
{
// On a Ctrl+C the interrupt watchdog owns the tree: keep the
// hook registered (forgetting the guard) so it removes the
// partial directory, instead of the historical behavior of
// leaving it in place. Without an interrupt this is a plain
// bootstrap failure and the partial directory stays, as before.
if crate::interrupt::interrupted()
&& let Some(hook) = cleanup_hook
{
std::mem::forget(hook);
} else {
drop(cleanup_hook);
}
return Err(e);
}
// Switch to an ephemeral context to build the package in the chroot.
// The parent is the base context itself (the one that bootstrapped
// the chroot), wired through `with_parent` instead of a config-name
// lookup, so an explicit non-current base (e.g. ssh) is used for
// everything that runs inside the chroot. The Arc stays in the
// guard: the build and the cleanup use it directly.
let ephemeral_ctx = Arc::new(Context::with_parent(
ContextConfig::Unshare {
path: chroot_path.to_string_lossy().to_string(),
parent: Some(current_context_name.clone()),
}));
// The real parent is bound below via `with_parent`; the
// config field is only used for contexts read from the
// persisted configuration.
parent: None,
},
base_ctx.clone(),
));
context::manager().set_current_ephemeral(ephemeral_ctx.clone());
Ok(Self {
previous_context: current_context_name,
previous_context,
ephemeral_ctx,
chroot_path,
build_succeeded: false,
base_ctx,
cleanup_hook,
})
}
/// The ephemeral build context created by this guard
///
/// Callers must take the context from here rather than from
/// [`crate::context::current()`]: the process-global is a shared swap
/// slot that another concurrent build may have re-pointed at its own
/// chroot, while this handle is guaranteed to be this guard's context.
pub fn context(&self) -> Arc<Context> {
Arc::clone(&self.ephemeral_ctx)
}
async fn download_and_extract_chroot(
series: &str,
arch: Option<&str>,
chroot_path: &PathBuf,
ctx: Arc<context::Context>,
ui: &Option<Arc<DebUi>>,
view: &dyn BuildView,
) -> Result<(), Box<dyn Error>> {
// Clone ctx for use in create_device_nodes after download_chroot_tarball consumes it
let ctx_for_devices = ctx.clone();
@@ -95,6 +346,11 @@ impl EphemeralContextGuard {
let poll_interval = 5; // Check every 5 seconds
while ctx.exists(&lockfile_path)? {
// Stop waiting on a Ctrl+C: the interrupt watchdog removes the
// (yet empty) chroot and exits without waiting for the poll
if crate::interrupt::interrupted() {
return Err("Interrupted while waiting for the chroot tarball".into());
}
if wait_time >= timeout {
log::warn!(
"Lockfile {} exists and has been present for more than {} seconds. \
@@ -123,10 +379,8 @@ impl EphemeralContextGuard {
series,
arch
);
if let Some(u) = ui {
u.phase(Phase::PreparingChroot);
}
Self::download_chroot_tarball(series, arch, &tarball_path, ctx, ui).await?;
enter_phase(view, Phase::PreparingChroot);
Self::download_chroot_tarball(series, arch, &tarball_path, ctx, view).await?;
} else {
log::debug!(
"Using cached chroot tarball for {} (arch: {:?})",
@@ -137,16 +391,12 @@ impl EphemeralContextGuard {
// Extract tarball to chroot directory
log::debug!("Extracting chroot tarball to {}...", chroot_path.display());
if let Some(u) = ui {
u.phase(Phase::ExtractingChroot);
}
Self::extract_tarball(&tarball_path, chroot_path, ui.as_deref())?;
enter_phase(view, Phase::ExtractingChroot);
Self::extract_tarball(&tarball_path, chroot_path, view)?;
// Create device nodes in the chroot
log::debug!("Creating device nodes in chroot...");
if let Some(u) = ui {
u.phase(Phase::FinalizingChroot);
}
enter_phase(view, Phase::FinalizingChroot);
Self::create_device_nodes(chroot_path, ctx_for_devices.clone())?;
// Bind mount /proc from host into chroot (before entering unshare namespace)
@@ -162,7 +412,7 @@ impl EphemeralContextGuard {
arch: Option<&str>,
tarball_path: &Path,
ctx: Arc<context::Context>,
ui: &Option<Arc<DebUi>>,
view: &dyn BuildView,
) -> Result<(), Box<dyn Error>> {
// Create a lock file to make sure that noone tries to use the file while it's not fully downloaded
let lockfile_path = tarball_path.with_extension("lock");
@@ -196,8 +446,8 @@ impl EphemeralContextGuard {
cmd.arg(series)
.arg(tarball_path.to_string_lossy().to_string());
if let Some(u) = ui {
cmd.capture(u.sink());
if let Some(s) = view.sink() {
cmd.capture(s);
}
let status = cmd.status()?;
@@ -234,7 +484,7 @@ impl EphemeralContextGuard {
fn extract_tarball(
tarball_path: &PathBuf,
chroot_path: &PathBuf,
ui: Option<&DebUi>,
view: &dyn BuildView,
) -> Result<(), Box<dyn Error>> {
// Create the chroot directory
fs::create_dir_all(chroot_path)?;
@@ -249,18 +499,19 @@ impl EphemeralContextGuard {
// too expensive for multi-hundred-MB chroot tarballs)
let mut count = 0usize;
for entry in archive.entries()? {
// Bail on a Ctrl+C before the interrupt watchdog's rm -rf races
// this loop writing entries into the tree being removed
if crate::interrupt::interrupted() {
return Err("Interrupted while extracting the chroot".into());
}
let mut entry = entry?;
entry.unpack_in(chroot_path)?;
count += 1;
if count.is_multiple_of(100)
&& let Some(u) = ui
{
u.progress_message(&format!("Extracting chroot… ({count} files)"));
if count.is_multiple_of(100) {
view.message(&format!("Extracting chroot… ({count} files)"));
}
}
if let Some(u) = ui {
u.progress_message(&format!("Extracting chroot… ({count} files)"));
}
view.message(&format!("Extracting chroot… ({count} files)"));
Ok(())
}
@@ -370,21 +621,47 @@ impl EphemeralContextGuard {
impl Drop for EphemeralContextGuard {
fn drop(&mut self) {
// On Ctrl+C the interrupt watchdog owns the chroot teardown through
// the registered hook: duplicating it here would race the hook's
// umount/rm (mounts vanish under each other). Dropping this guard
// would normally deregister the hook, so while the watchdog runs it
// must be leaked instead to keep it registered (if it was already
// drained, forgetting is a harmless no-op).
if crate::interrupt::interrupted() {
context::manager().set_current_ephemeral(self.previous_context.clone());
if let Some(hook) = self.cleanup_hook.take() {
std::mem::forget(hook);
}
return;
}
// Deregister the interrupt-time cleanup hook first: the normal
// cleanup below takes care of the chroot, so the hook must not fire
// afterwards. (If a SIGINT arrived mid-drop and the hook is already
// running concurrently, deregistration simply does not find it —
// both paths are individually idempotent and failure-tolerant.)
if let Some(mut cleanup_hook) = self.cleanup_hook.take() {
cleanup_hook.deregister();
}
log::debug!("Cleaning up ephemeral context ({:?})...", self.chroot_path);
// Clean up any overlay mounts before resetting the context.
// This must happen while the ephemeral context is still current so its
// driver is accessible. The actual unmount commands run via the parent
// Clean up any overlay mounts before resetting the context. This
// explicitly targets the context this guard created — never
// `context::current()`, which a concurrent build may have re-pointed
// at its own chroot. The actual unmount commands run via the parent
// (base) context, so they work regardless.
let ephemeral_ctx = context::current();
if let Err(e) = ephemeral_ctx.cleanup() {
if let Err(e) = self.ephemeral_ctx.cleanup() {
log::warn!("Failed to clean up overlay mounts: {}", e);
}
// Reset to normal context
if let Err(e) = context::manager().set_current(&self.previous_context) {
log::error!("Failed to restore context {}: {}", self.previous_context, e);
}
// Restore the context that was current when this guard was created,
// not whatever is globally current at drop time (another concurrent
// build's override may be installed there). This only swaps the
// in-memory handle: the persisted configuration still names the
// context selected by the user, as `set_current_ephemeral` never
// touches it.
context::manager().set_current_ephemeral(self.previous_context.clone());
// Remove chroot directory only if build succeeded
if self.build_succeeded {
@@ -453,3 +730,58 @@ impl Drop for EphemeralContextGuard {
}
}
}
#[cfg(test)]
mod chroot_cleanup_tests {
use super::*;
/// /proc/mounts path fields use octal escapes for whitespace and
/// backslashes; anything else must be kept verbatim.
#[test]
fn mount_field_unescaping_decodes_octal_escapes() {
assert_eq!(unescape_mount_field("/mnt/plain"), "/mnt/plain");
assert_eq!(
unescape_mount_field("/mnt/with\\040space"),
"/mnt/with space"
);
assert_eq!(unescape_mount_field("/mnt/with\\011tab"), "/mnt/with\ttab");
assert_eq!(unescape_mount_field("back\\134slash"), "back\\slash");
// Not an escape sequence: kept verbatim
assert_eq!(unescape_mount_field("back\\9slash"), "back\\9slash");
assert_eq!(unescape_mount_field("trailing\\"), "trailing\\");
}
/// Interrupt cleanup of a path that has no mounts and does not exist must
/// be a harmless no-op (no panic, nothing left behind).
#[test]
fn sigint_cleanup_of_missing_chroot_is_a_noop() {
let dir = tempfile::tempdir().unwrap();
let missing = dir.path().join("no-such-chroot");
sigint_cleanup_chroot(&missing);
assert!(!missing.exists());
}
/// A real directory with no mounts under it is simply removed. Skipped
/// when non-root without working non-interactive sudo, since removal then
/// legitimately fails (and is only logged).
#[test]
fn sigint_cleanup_removes_an_unmounted_directory() {
let is_root = unsafe { libc::geteuid() } == 0;
if !is_root
&& !privileged_command("true", false)
.status()
.is_ok_and(|s| s.success())
{
return;
}
let dir = tempfile::tempdir().unwrap();
let chroot = dir.path().join("chroot");
std::fs::create_dir_all(chroot.join("rootfs")).unwrap();
std::fs::write(chroot.join("rootfs").join("file.txt"), "data").unwrap();
sigint_cleanup_chroot(&chroot);
assert!(!chroot.exists());
}
}
+880 -137
View File
File diff suppressed because it is too large Load Diff
+688 -125
View File
@@ -1,9 +1,15 @@
mod cross;
mod ephemeral;
/// Ephemeral (per-build) unshare contexts, including the process-global
/// cleanup-hook registry drained by the SIGINT handler
pub(crate) mod ephemeral;
mod local;
use crate::context::{self, Context};
use crate::ui::deb::{DebUi, Phase};
use crate::logfmt::{
AptInstallClassifier, AptUpdateClassifier, Classifier, GenericClassifier, MakeClassifier,
MmdebstrapClassifier, QuiltClassifier,
};
use crate::report::{BuildTarget, BuildView};
use std::error::Error;
use std::path::{Path, PathBuf};
use std::sync::Arc;
@@ -15,110 +21,225 @@ pub enum BuildMode {
Local,
}
/// Phases of a binary build, announced to the [`BuildView`]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Phase {
/// Downloading the chroot tarball (mmdebstrap)
PreparingChroot,
/// Extracting the chroot tarball
ExtractingChroot,
/// Device nodes, /proc bind mount, etc.
FinalizingChroot,
/// apt-get update
UpdatingPackageLists,
/// Installing build-essential & co
InstallingEssentials,
/// quilt push -a
ApplyingPatches,
/// --inject packages
InjectingPackages,
/// apt-get build-dep
InstallingBuildDeps,
/// debian/rules build
Building,
/// fakeroot debian/rules binary
ProducingBinaries,
/// Retrieving produced .deb files
RetrievingArtifacts,
}
impl Phase {
/// Human-readable label displayed in the status bar
pub fn label(&self) -> &'static str {
match self {
Phase::PreparingChroot => "Preparing chroot",
Phase::ExtractingChroot => "Extracting chroot",
Phase::FinalizingChroot => "Finalizing chroot",
Phase::UpdatingPackageLists => "Updating package lists",
Phase::InstallingEssentials => "Installing essential packages",
Phase::ApplyingPatches => "Applying patches",
Phase::InjectingPackages => "Injecting packages",
Phase::InstallingBuildDeps => "Installing build dependencies",
Phase::Building => "Building package",
Phase::ProducingBinaries => "Producing binary packages",
Phase::RetrievingArtifacts => "Retrieving artifacts",
}
}
}
/// Default line classifier rewriting a phase's subprocess output
fn default_classifier(phase: Phase) -> Box<dyn Classifier> {
match phase {
Phase::PreparingChroot => Box::new(MmdebstrapClassifier::new()),
Phase::ExtractingChroot | Phase::FinalizingChroot => Box::new(GenericClassifier::new()),
Phase::UpdatingPackageLists => Box::new(AptUpdateClassifier::new()),
Phase::InstallingEssentials => Box::new(AptInstallClassifier::new("Installing essentials")),
Phase::ApplyingPatches => Box::new(QuiltClassifier::new(0)),
Phase::InjectingPackages => Box::new(AptInstallClassifier::new("Injecting packages")),
Phase::InstallingBuildDeps => {
Box::new(AptInstallClassifier::new("Installing build dependencies"))
}
Phase::Building | Phase::ProducingBinaries => Box::new(MakeClassifier::new()),
Phase::RetrievingArtifacts => Box::new(GenericClassifier::new()),
}
}
/// Enter `phase` on the view with its default line classifier
pub(crate) fn enter_phase(view: &dyn BuildView, phase: Phase) {
view.phase(phase.label(), default_classifier(phase));
}
/// Parameters of one [`build_binary_package`] call.
pub struct DebBuildOptions<'a> {
/// Target architecture; defaults to the host architecture.
pub arch: Option<String>,
/// Target distribution series; defaults to the changelog series
/// (UNRELEASED resolves to the vendor's development series).
pub series: Option<String>,
/// Distribution pocket to resolve build-dependencies from.
pub pocket: Option<String>,
/// Source tree to build; defaults to the process working directory.
pub cwd: Option<PathBuf>,
/// Cross-compile for the target architecture instead of using
/// qemu-binfmt.
pub cross: bool,
/// Build mode; defaults to [`BuildMode::Local`].
pub mode: Option<BuildMode>,
/// PPAs to add for build-dependencies (`user/ppa_name`).
pub ppa: Vec<String>,
/// Packages to inject into the build environment before build-dep
/// (.deb paths, archive names or PPA packages).
pub inject: Vec<String>,
/// Parallel build jobs; defaults to the core count available in the
/// build context.
pub jobs: Option<usize>,
/// Explicit build context; defaults to the current context.
pub ctx: Option<Arc<Context>>,
/// Where build events (phases, progress, outcome) are reported.
pub view: &'a dyn BuildView,
}
impl Default for DebBuildOptions<'_> {
fn default() -> Self {
static QUIET: crate::report::Quiet = crate::report::Quiet;
DebBuildOptions {
arch: None,
series: None,
pocket: None,
cwd: None,
cross: false,
mode: None,
ppa: Vec::new(),
inject: Vec::new(),
jobs: None,
ctx: None,
view: &QUIET,
}
}
}
/// Build package in 'cwd' to a .deb
///
/// Returns the list of produced .deb files retrieved locally. When `ui` is
/// set, a live view (status bar + rolling log pane) is displayed and all
/// subprocess output is captured through it; on failure the widget is cleared
/// and a summary of captured errors is printed.
#[allow(clippy::too_many_arguments)]
/// Returns the list of produced artifacts (.deb files plus the upload
/// metadata `.buildinfo`/`.changes`) retrieved locally, identified from
/// `debian/files` and the native metadata generation rather than by
/// globbing the build root (which would surface stale files). Subprocess
/// output is captured through the view's sink (live view + tee log for the
/// terminal adapter); on failure the view is cleared and prints a summary
/// of captured errors.
pub async fn build_binary_package(
arch: Option<&str>,
series: Option<&str>,
pocket: Option<&str>,
cwd: Option<&Path>,
cross: bool,
mode: Option<BuildMode>,
ppa: Option<&[&str]>,
inject_packages: Option<&[&str]>,
ctx: Option<Arc<Context>>,
ui: Option<Arc<DebUi>>,
opts: DebBuildOptions<'_>,
) -> Result<Vec<PathBuf>, Box<dyn Error>> {
let result = build_binary_package_impl(
arch,
series,
pocket,
cwd,
cross,
mode,
ppa,
inject_packages,
ctx,
&ui,
)
.await;
let view = opts.view;
let result = build_binary_package_impl(opts).await;
if let (Some(u), Err(_)) = (&ui, &result) {
u.finish_failure();
if result.is_err() {
view.finish_failure();
}
result
}
/// Implementation of [`build_binary_package`], without failure handling
#[allow(clippy::too_many_arguments)]
async fn build_binary_package_impl(
arch: Option<&str>,
series: Option<&str>,
pocket: Option<&str>,
cwd: Option<&Path>,
cross: bool,
mode: Option<BuildMode>,
ppa: Option<&[&str]>,
inject_packages: Option<&[&str]>,
ctx: Option<Arc<Context>>,
ui: &Option<Arc<DebUi>>,
opts: DebBuildOptions<'_>,
) -> Result<Vec<PathBuf>, Box<dyn Error>> {
let cwd = cwd.unwrap_or_else(|| Path::new("."));
let DebBuildOptions {
ref arch,
ref series,
ref pocket,
ref cwd,
cross,
ref mode,
ref ppa,
ref inject,
ref jobs,
ref ctx,
view,
} = opts;
let cwd = cwd.as_deref().unwrap_or_else(|| Path::new("."));
// Parse changelog to get package name, version and series
let changelog_path = cwd.join("debian/changelog");
let (package, version, package_series) =
crate::changelog::parse_changelog_header(&changelog_path)?;
// UNRELEASED is not a real archive series: without an explicit --series,
// build against the development series of the host vendor's distribution
// instead. An explicit --series always wins.
let resolved_series;
let series = if let Some(s) = series {
s
} else if crate::distro_info::is_unreleased(&package_series) {
let dist = crate::build::env::current_vendor();
resolved_series = crate::distro_info::effective_series(&package_series, &dist).await?;
log::info!(
"Changelog is UNRELEASED, building against series {}",
resolved_series
);
resolved_series.as_str()
} else {
&package_series
};
let current_arch = crate::get_current_arch();
let arch = arch.unwrap_or(&current_arch);
let arch = arch.as_deref().unwrap_or(&current_arch);
// Make sure we select a specific mode, either using user-requested
// or by using default for user-supplied parameters
let mode = if let Some(m) = mode {
m
} else {
// By default, we use local build
BuildMode::Local
};
let default_mode = BuildMode::Local;
let mode = mode.as_ref().unwrap_or(&default_mode);
// Create an ephemeral unshare context for all Local builds
// Use qemu_binfmt when target architecture differs from host and cross is not requested
let chroot_arch = if mode == BuildMode::Local && arch != current_arch && !cross {
let chroot_arch = if mode == &BuildMode::Local && arch != current_arch && !cross {
Some(arch)
} else {
None
};
// Use provided context or get current
let base_ctx = ctx.unwrap_or_else(context::current);
let base_ctx = ctx.clone().unwrap_or_else(context::current);
// Identify the target in the live UI once the changelog is parsed, so
// Identify the target in the live view once the changelog is parsed, so
// even the chroot download output is attributed and tee'd
if let Some(u) = ui {
u.set_target(&package, &version, series, arch);
}
view.target(BuildTarget {
package: &package,
version: &version,
target: &format!("{series}/{arch}"),
display: format!("Building {package} ({version}) for {series}/{arch}"),
source_only: false,
tee_log: true,
});
// Create an ephemeral unshare context for all Local builds. It is kept in
// this scope so it outlives the guarded section below and is only dropped
// once the live view has been cleared.
let mut guard = if mode == BuildMode::Local {
let mut guard = if *mode == BuildMode::Local {
Some(
ephemeral::EphemeralContextGuard::new_with_context(
series,
chroot_arch,
base_ctx.clone(),
ui.clone(),
view,
)
.await?,
)
@@ -126,14 +247,17 @@ async fn build_binary_package_impl(
None
};
let result = async {
// Get the build context - either the ephemeral context or the base context
let build_ctx = if mode == BuildMode::Local {
context::current()
} else {
base_ctx.clone()
// Determine the build context explicitly: for Local builds it is the
// ephemeral context the guard just created (taken from the guard itself,
// never from the process-global, which concurrent builds may have
// re-pointed at their own chroot); otherwise the base context is used
// directly.
let build_ctx = match guard.as_ref() {
Some(g) => g.context(),
None => base_ctx.clone(),
};
let result = async {
// Prepare build directory
let build_root = build_ctx.create_temp_dir()?;
@@ -145,60 +269,60 @@ async fn build_binary_package_impl(
.ok_or("Cannot find parent directory name")?;
let build_root = format!("{}/{}", build_root, parent_dir_name.to_str().unwrap());
// Run the build using target build mode
match mode {
// Resolve the package directory inside the staging area. The tree
// the caller pointed at is authoritative (its changelog defined the
// package/version/series above), so its staged copy wins; the
// name-pattern search only runs as a fallback.
let package_dir = resolve_package_directory(
Path::new(&build_root),
cwd,
&package,
&version,
series,
&build_ctx,
)?;
// Run the build using target build mode. It returns the exact set of
// artifacts produced by this build (binary packages registered in
// debian/files plus the generated .buildinfo/.changes), as paths
// inside the build context.
let remote_files: Vec<PathBuf> = match mode {
BuildMode::Local => {
local::build(
&package,
&version,
arch,
series,
pocket,
pocket.as_deref(),
&build_root,
&package_dir,
cross,
ppa,
inject_packages,
inject,
build_ctx.clone(),
ui.clone(),
view,
*jobs,
)
.await?
}
}
};
// Retrieve produced artifacts (.deb files plus the upload metadata
// (.buildinfo/.changes) generated natively after the build)
if let Some(u) = ui {
u.phase(Phase::RetrievingArtifacts);
}
let remote_files = build_ctx.list_files(Path::new(&build_root))?;
let deb_files: Vec<PathBuf> = remote_files
.into_iter()
.filter(|f| {
f.extension().is_some_and(|ext| {
matches!(
ext.to_str(),
Some("deb") | Some("buildinfo") | Some("changes")
)
})
})
.collect();
let total_debs = deb_files.len();
// Retrieve the produced artifacts (binary packages plus the upload
// metadata) to the parent directory.
enter_phase(view, Phase::RetrievingArtifacts);
let total_debs = remote_files.len();
let mut artifacts = Vec::with_capacity(total_debs);
for (idx, remote_file) in deb_files.iter().enumerate() {
for (idx, remote_file) in remote_files.iter().enumerate() {
let file_name = remote_file.file_name().ok_or("Invalid remote filename")?;
let local_dest = parent_dir.join(file_name);
build_ctx.retrieve_path(remote_file, &local_dest)?;
artifacts.push(local_dest);
if let Some(u) = ui {
u.count_progress("Retrieving artifacts", idx + 1, total_debs);
}
view.progress("Retrieving artifacts", idx + 1, total_debs);
}
if let Some(u) = ui {
u.finish_success(&artifacts, u.elapsed());
}
view.finish_success(&artifacts);
Ok(artifacts)
}
@@ -207,9 +331,7 @@ async fn build_binary_package_impl(
// Clear the live view before returning: the ephemeral guard is dropped at
// the end of this function and its cleanup commands (umount, rm -rf of
// the chroot) inherit the terminal, so they must not fight the widget.
if let Some(u) = ui {
u.suspend();
}
view.suspend();
// Mark build as successful to trigger chroot cleanup
if result.is_ok()
@@ -221,6 +343,38 @@ async fn build_binary_package_impl(
result
}
/// Resolve the package directory for a build inside the staged build root.
///
/// The tree the caller pointed at is authoritative: `cwd`'s changelog
/// already defined the package, version and series for this build, so its
/// staged copy is used outright when it carries a `debian/` tree. The
/// name-pattern search ([`find_package_directory`], including the quirks
/// overrides) only runs when that copy cannot be resolved — a default `.`
/// cwd has no basename, and the pointed-at tree may live outside the staged
/// parent. Embedded callers are the motivation: their working directory
/// names (`tree`, `checkout`, ...) match none of the search patterns.
pub(crate) fn resolve_package_directory(
build_root: &Path,
cwd: &Path,
package: &str,
version: &str,
series: &str,
ctx: &context::Context,
) -> Result<PathBuf, Box<dyn Error>> {
if let Some(tree_name) = cwd.file_name() {
let staged_tree = build_root.join(tree_name);
if ctx.is_dir(&staged_tree)? && ctx.exists(&staged_tree.join("debian"))? {
log::debug!(
"Using the staged copy of {} at {}",
cwd.display(),
staged_tree.display()
);
return Ok(staged_tree);
}
}
find_package_directory(build_root, package, version, series, ctx)
}
/// Find the current package directory by trying both patterns:
/// - package/package
/// - package/package-origversion
@@ -229,10 +383,11 @@ pub(crate) fn find_package_directory(
parent_dir: &Path,
package: &str,
version: &str,
series: &str,
ctx: &context::Context,
) -> Result<PathBuf, Box<dyn Error>> {
// Check quirks first for custom package directories
let custom_dirs = crate::quirks::get_package_directories(package);
let custom_dirs = crate::quirks::get_package_directories(package, series);
for custom_dir in custom_dirs {
let package_dir = parent_dir.join(&custom_dir);
if ctx.exists(&package_dir)? && ctx.exists(&package_dir.join("debian"))? {
@@ -304,11 +459,13 @@ pub(crate) fn find_package_directory(
let entries = ctx.list_files(package_parent)?;
let mut found_dirs = Vec::new();
for entry in entries {
if entry.is_dir() {
if let Some(file_name) = entry.file_name() {
found_dirs.push(file_name.to_string_lossy().into_owned());
}
// list_files yields context-relative paths (e.g. rooted inside
// the chroot for an unshare context): classify through the
// context, a host-side stat would miss every entry.
let is_dir = ctx.is_dir(&entry)?;
log::debug!(" - {}", entry.display());
if is_dir && let Some(file_name) = entry.file_name() {
found_dirs.push(file_name.to_string_lossy().into_owned());
}
}
@@ -351,12 +508,15 @@ fn find_dsc_file(
}
/// Check whether an apt source URI points to a distribution archive
/// (as opposed to a PPA or another third-party repository)
/// (as opposed to a PPA or another third-party repository): a thin,
/// dist-agnostic wrapper over [`crate::distro_info::is_official_source`],
/// unioned across every known distro. Where the distribution is known,
/// the dist-scoped check is preferred (it cannot misfire on another
/// distro's mirror); this fallback stays for the sites that cannot know.
pub(crate) fn is_archive_source(uri: &str) -> bool {
uri.contains("archive.ubuntu.com")
|| uri.contains("security.ubuntu.com")
|| uri.contains("ports.ubuntu.com")
|| uri.contains("deb.debian.org")
crate::distro_info::supported_dists()
.iter()
.any(|dist| crate::distro_info::is_official_source(dist, uri))
}
#[cfg(test)]
@@ -364,6 +524,126 @@ mod tests {
use super::*;
use std::sync::Arc;
/// The archive check is dist-agnostic (any distro's official mirror
/// counts) and host-based: the country mirrors the old substring
/// checks matched (`fr.archive.ubuntu.com`) still count, look-alike
/// hosts and PPAs do not.
#[test]
fn archive_sources_are_official_mirrors_of_any_distro() {
for uri in [
"https://archive.ubuntu.com/ubuntu",
// Country mirrors front the Ubuntu archive.
"http://fr.archive.ubuntu.com/ubuntu",
"http://security.ubuntu.com/ubuntu",
"http://ports.ubuntu.com/ubuntu-ports",
"https://deb.debian.org/debian",
] {
assert!(is_archive_source(uri), "{uri}");
}
for uri in [
"https://ppa.launchpadcontent.net/user/ppa/ubuntu",
"http://notarchive.ubuntu.com/ubuntu",
"https://example.com/debian",
] {
assert!(!is_archive_source(uri), "{uri}");
}
}
/// An unshare context mapped over `chroot_root`, parented on a local
/// context like the ephemeral build contexts are: exists/list_files/
/// is_dir answer through the path mapping, no namespace privileges
/// needed.
fn unshare_test_context(chroot_root: &Path) -> Context {
let base = Context::new(crate::context::ContextConfig::Local).unwrap();
Context::with_parent(
crate::context::ContextConfig::Unshare {
path: chroot_root.to_string_lossy().to_string(),
parent: None,
},
Arc::new(base),
)
}
/// The staging-area listing must classify entries through the context:
/// an unshare context returns build-root-relative paths that a host-side
/// stat never sees (they live under the chroot root on the host), which
/// used to silently empty the 'Found directories' list of the search
/// failure message — and with it every hint about the actual layout.
#[test]
fn find_package_directory_lists_staged_directories_through_the_context() {
let chroot = tempfile::tempdir().unwrap();
// Staged parent holding a single tree whose name matches none of
// the search patterns (the embedded-caller layout: <job>/tree)
let staged_parent = chroot.path().join("tmp/pkh-build-1/j-42");
std::fs::create_dir_all(staged_parent.join("tree/debian")).unwrap();
let ctx = unshare_test_context(chroot.path());
let err = find_package_directory(
Path::new("/tmp/pkh-build-1/j-42"),
"bc",
"1.07.1-1ubuntu1",
"questing",
&ctx,
)
.expect_err("no candidate matches a tree named 'tree'");
let message = err.to_string();
assert!(
message.contains("Found directories: tree"),
"error should list the staged directories through the context: {message}"
);
}
/// An explicit cwd must resolve to its staged copy even when its name
/// matches none of the search patterns: the pointed-at tree is what the
/// parsed changelog came from.
#[test]
fn resolve_package_directory_prefers_the_pointed_tree() {
let chroot = tempfile::tempdir().unwrap();
let staged_parent = chroot.path().join("tmp/pkh-build-1/j-42");
std::fs::create_dir_all(staged_parent.join("tree/debian/source")).unwrap();
let ctx = unshare_test_context(chroot.path());
let resolved = resolve_package_directory(
Path::new("/tmp/pkh-build-1/j-42"),
Path::new("/work/jobs/j-42/tree"),
"bc",
"1.07.1-1ubuntu1",
"questing",
&ctx,
)
.expect("the staged copy of the pointed-at tree must resolve");
assert_eq!(resolved, PathBuf::from("/tmp/pkh-build-1/j-42/tree"));
}
/// When the pointed-at tree is not in the staging area under its own
/// name, resolution falls back to the name-pattern search.
#[test]
fn resolve_package_directory_falls_back_to_the_name_search() {
let chroot = tempfile::tempdir().unwrap();
let staged_parent = chroot.path().join("tmp/pkh-build-1/j-42");
// Staged copy of a pulled tree: <pkg>/<pkg>-<origversion>
std::fs::create_dir_all(staged_parent.join("bc/bc-1.07.1/debian")).unwrap();
let ctx = unshare_test_context(chroot.path());
let resolved = resolve_package_directory(
Path::new("/tmp/pkh-build-1/j-42"),
// A tree never staged under that name
Path::new("/work/other/checkout"),
"bc",
"1.07.1-1ubuntu1",
"questing",
&ctx,
)
.expect("the pulled-tree layout must resolve via the name search");
assert_eq!(
resolved,
PathBuf::from("/tmp/pkh-build-1/j-42/bc/bc-1.07.1")
);
}
async fn test_build_end_to_end(
package: &str,
series: &str,
@@ -385,7 +665,7 @@ mod tests {
log::info!("Pulling package {} from {}...", package, series);
let package_info =
crate::package_info::lookup(package, None, Some(series), "", dist, None, None)
crate::package_info::lookup(package, None, Some(series), "", dist, None, None, None)
.await
.expect("Cannot lookup package information");
crate::pull::pull(&package_info, Some(cwd), None, true)
@@ -394,27 +674,28 @@ mod tests {
log::info!("Successfully pulled package {}", package);
// Create a fresh local context for this test
let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local));
let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local).unwrap());
// Change directory to the package directory
let cwd =
crate::deb::find_package_directory(cwd, package, &package_info.stanza.version, &ctx)
let cwd = crate::deb::find_package_directory(
cwd,
package,
&package_info.stanza.version,
series,
&ctx,
)
.expect("Cannot find package directory");
log::debug!("Package directory: {}", cwd.display());
log::info!("Starting binary package build...");
crate::deb::build_binary_package(
arch,
Some(series),
None,
Some(&cwd),
crate::deb::build_binary_package(DebBuildOptions {
arch: arch.map(str::to_string),
series: Some(series.to_string()),
cross,
None,
None,
None,
Some(ctx),
None,
)
cwd: Some(cwd.to_path_buf()),
ctx: Some(ctx),
..Default::default()
})
.await
.expect("Cannot build binary package (deb)");
log::info!("Successfully built binary package");
@@ -471,12 +752,34 @@ mod tests {
/// NOTE: Ideally, we want to run this in CI, but it takes more than 1h
/// to fully build the linux-riscv package on an amd64 builder, which is too
/// much time
/// The series is the current LTS (26.04) rather than an interim one:
/// interim series vanish from the mirrors a few months after their EOL
/// (questing is already unreachable), an LTS stays pullable for years.
#[ignore]
#[tokio::test]
#[test_log::test]
#[cfg(target_arch = "x86_64")]
async fn test_deb_linux_riscv_ubuntu_cross_end_to_end() {
test_build_end_to_end("linux-riscv", "questing", None, Some("riscv64"), true).await;
test_build_end_to_end("linux-riscv", "resolute", None, Some("riscv64"), true).await;
}
/// KNOWN-BROKEN cross build of the noble-era kernel, kept as an
/// ignored fixture to work from. Noble controls declare their build
/// tools unqualified (the `:native` idiom landed later), so exact dpkg
/// semantics demand host-architecture instances of them
/// (python3:riscv64, gcc-13:riscv64, clang-17:riscv64, ...) and the
/// resulting two-architecture install set is unsolvable: t64
/// libraries (libclang1-17t64) conflict with their own foreign-arch
/// variant, and the riscv64 toolchain instances drag depends chains
/// (gcc:riscv64) that do not resolve from the chroot sources. A real
/// run fails at the apt transaction with 'Unable to correct problems'.
/// Resolute-era controls declare :native properly; see the test above.
#[ignore]
#[tokio::test]
#[test_log::test]
#[cfg(target_arch = "x86_64")]
async fn test_deb_linux_riscv_noble_cross_end_to_end() {
test_build_end_to_end("linux-riscv", "noble", None, Some("riscv64"), true).await;
}
/// This is a specific test case for the latest gcc package on Debian
@@ -493,4 +796,264 @@ mod tests {
async fn test_deb_gcc_debian_end_to_end() {
test_build_end_to_end("gcc-15", "sid", None, None, false).await;
}
/// Create a synthetic source package that discriminates which architecture
/// is used to resolve Build-Depends-Indep during cross builds:
///
/// - 'libdb-dev' is an arch:any package that is not Multi-Arch: same, so an
/// amd64 copy can only be installed by replacing the arm64 one
/// - the arch-specific binary links against libdb for the host
/// architecture, so the build only succeeds if the arm64 libdb-dev was
/// left in place by the arch-independant build-dep pass
fn create_indep_cross_test_source(parent: &Path) -> PathBuf {
let pkg_dir = parent.join("pkh-crosstest");
std::fs::create_dir_all(pkg_dir.join("debian/source")).unwrap();
std::fs::write(
pkg_dir.join("debian/changelog"),
"pkh-crosstest (1.0) noble; urgency=medium\n\n \
* Synthetic package exercising Build-Depends-Indep in cross builds.\n\n \
-- pkh tests <pkh@example.com> Tue, 15 Sep 2026 08:00:00 +0000\n",
)
.unwrap();
std::fs::write(
pkg_dir.join("debian/control"),
"Source: pkh-crosstest\n\
Section: devel\n\
Priority: optional\n\
Maintainer: pkh tests <pkh@example.com>\n\
Standards-Version: 4.7.4\n\
Build-Depends: debhelper-compat (= 13), libdb-dev\n\
Build-Depends-Indep: libdb-dev\n\
Architecture: any all\n\
\n\
Package: pkh-crosstest\n\
Architecture: any\n\
Depends: ${misc:Depends}, ${shlibs:Depends}\n\
Description: Cross-build regression package for build-dep resolution\n \
Builds a host-architecture binary against libdb to detect a cross\n \
build environment damaged by a wrongly-scoped build-dep pass.\n\
\n\
Package: pkh-crosstest-data\n\
Architecture: all\n\
Description: Cross-build regression package data (arch-indep)\n \
Arch-indep binary so the indep build path is exercised.\n",
)
.unwrap();
std::fs::write(
pkg_dir.join("debian/rules"),
"#!/usr/bin/make -f\n\
%:\n\
\tdh $@\n\
\n\
override_dh_auto_build:\n\
\tprintf '#include <db.h>\\nint main(void){DB *d; return db_create(&d, NULL, 0);}\\n' > main.c\n\
\t$(DEB_HOST_GNU_TYPE)-gcc main.c -ldb -o pkh-crosstest\n",
)
.unwrap();
use std::os::unix::fs::PermissionsExt;
let rules = pkg_dir.join("debian/rules");
let mut perms = std::fs::metadata(&rules).unwrap().permissions();
perms.set_mode(0o755);
std::fs::set_permissions(&rules, perms).unwrap();
std::fs::write(pkg_dir.join("debian/source/format"), "3.0 (native)\n").unwrap();
pkg_dir
}
/// This ensures the arch-independant build-dep pass of a cross build
/// resolves dependencies for the host architecture, like dpkg-checkbuilddeps
/// does, instead of re-resolving the whole Build-Depends field for the
/// native architecture, which swaps host-arch -dev packages for native ones
/// and breaks the cross build environment.
#[tokio::test]
#[test_log::test]
#[cfg(target_arch = "x86_64")]
async fn test_deb_cross_indep_host_arch_end_to_end() {
let temp_dir = tempfile::tempdir().unwrap();
let pkg_dir = create_indep_cross_test_source(temp_dir.path());
let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local).unwrap());
crate::deb::build_binary_package(DebBuildOptions {
arch: Some("arm64".to_string()),
series: Some("noble".to_string()),
cwd: Some(pkg_dir),
cross: true,
ctx: Some(ctx),
..Default::default()
})
.await
.expect("Cannot cross-build package declaring Build-Depends-Indep");
// Both binary packages must have been produced, including the
// arch-independant one
let deb_files: Vec<String> = std::fs::read_dir(temp_dir.path())
.unwrap()
.filter_map(|e| e.ok())
.map(|e| e.file_name().to_string_lossy().to_string())
.collect();
assert!(
deb_files
.iter()
.any(|f| f.starts_with("pkh-crosstest_1.0_arm64.deb")),
"arch-specific .deb not produced, got: {deb_files:?}"
);
assert!(
deb_files
.iter()
.any(|f| f.starts_with("pkh-crosstest-data_1.0_all.deb")),
"arch-independant .deb not produced, got: {deb_files:?}"
);
}
/// A cross build whose build-dependencies cannot be satisfied under
/// dpkg's cross semantics must abort at the install step, naming the
/// unsatisfied dependency: the expected-failure counterpart of the
/// cross builds above.
#[tokio::test]
#[test_log::test]
#[cfg(target_arch = "x86_64")]
async fn test_deb_cross_unsatisfiable_build_dep_fails_end_to_end() {
let temp_dir = tempfile::tempdir().unwrap();
let pkg_dir = temp_dir.path().join("pkh-crosstest");
std::fs::create_dir_all(pkg_dir.join("debian/source")).unwrap();
std::fs::write(
pkg_dir.join("debian/changelog"),
"pkh-crosstest (1.0) noble; urgency=medium\n\n \
* Synthetic package declaring an unsatisfiable build-dependency.\n\n \
-- pkh tests <pkh@example.com> Tue, 15 Sep 2026 08:00:00 +0000\n",
)
.unwrap();
std::fs::write(
pkg_dir.join("debian/control"),
"Source: pkh-crosstest\n\
Section: devel\n\
Priority: optional\n\
Maintainer: pkh tests <pkh@example.com>\n\
Standards-Version: 4.7.4\n\
Build-Depends: pkh-no-such-package-xyz\n\
Architecture: any\n\
\n\
Package: pkh-crosstest\n\
Architecture: any\n\
Depends: ${misc:Depends}, ${shlibs:Depends}\n\
Description: Cross-build negative regression package\n \
Declares a build-dependency absent from the archive.\n",
)
.unwrap();
std::fs::write(
pkg_dir.join("debian/rules"),
"#!/usr/bin/make -f\n%:\n\tdh $@\n",
)
.unwrap();
use std::os::unix::fs::PermissionsExt;
let rules = pkg_dir.join("debian/rules");
let mut perms = std::fs::metadata(&rules).unwrap().permissions();
perms.set_mode(0o755);
std::fs::set_permissions(&rules, perms).unwrap();
std::fs::write(pkg_dir.join("debian/source/format"), "3.0 (native)\n").unwrap();
let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local).unwrap());
let err = crate::deb::build_binary_package(DebBuildOptions {
arch: Some("arm64".to_string()),
series: Some("noble".to_string()),
cwd: Some(pkg_dir),
cross: true,
ctx: Some(ctx),
..Default::default()
})
.await
.expect_err("an unsatisfiable build-dependency must fail the cross build");
assert!(
err.to_string().contains("pkh-no-such-package-xyz"),
"error should name the unsatisfied dependency: {err}"
);
}
/// An embedded-caller layout — the tree checked out at <job>/tree, a
/// name matching none of the search patterns — must build: the staged
/// copy of the tree the caller pointed at is resolved directly instead
/// of being re-derived from package/version names (which used to fail
/// with 'Could not find package directory').
#[tokio::test]
#[test_log::test]
async fn test_deb_builds_a_tree_named_directory_end_to_end() {
let temp_dir = tempfile::tempdir().unwrap();
let pkg_dir = temp_dir.path().join("j-42/tree");
std::fs::create_dir_all(pkg_dir.join("debian/source")).unwrap();
std::fs::write(
pkg_dir.join("debian/changelog"),
"pkh-treetest (1.0) noble; urgency=medium\n\n \
* Synthetic package built from a directory named 'tree'.\n\n \
-- pkh tests <pkh@example.com> Tue, 15 Sep 2026 08:00:00 +0000\n",
)
.unwrap();
std::fs::write(
pkg_dir.join("debian/control"),
"Source: pkh-treetest\n\
Section: devel\n\
Priority: optional\n\
Maintainer: pkh tests <pkh@example.com>\n\
Standards-Version: 4.7.4\n\
Build-Depends: debhelper-compat (= 13)\n\
Architecture: any\n\
\n\
Package: pkh-treetest\n\
Architecture: any\n\
Depends: ${misc:Depends}, ${shlibs:Depends}\n\
Description: Package-directory resolution regression package\n \
Its tree lives in a directory whose name matches none of the\n \
package-directory search patterns.\n",
)
.unwrap();
std::fs::write(
pkg_dir.join("debian/rules"),
"#!/usr/bin/make -f\n%:\n\tdh $@\n",
)
.unwrap();
use std::os::unix::fs::PermissionsExt;
let rules = pkg_dir.join("debian/rules");
let mut perms = std::fs::metadata(&rules).unwrap().permissions();
perms.set_mode(0o755);
std::fs::set_permissions(&rules, perms).unwrap();
std::fs::write(pkg_dir.join("debian/source/format"), "3.0 (native)\n").unwrap();
let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local).unwrap());
crate::deb::build_binary_package(DebBuildOptions {
series: Some("noble".to_string()),
cwd: Some(pkg_dir),
ctx: Some(ctx),
..Default::default()
})
.await
.expect("a tree named 'tree' must build");
let deb_files: Vec<String> = std::fs::read_dir(temp_dir.path().join("j-42"))
.unwrap()
.filter_map(|e| e.ok())
.map(|e| e.file_name().to_string_lossy().to_string())
.collect();
assert!(
deb_files
.iter()
.any(|f| f.starts_with("pkh-treetest_1.0_") && f.ends_with(".deb")),
".deb not produced for the 'tree'-named directory, got: {deb_files:?}"
);
}
}
+167 -41
View File
@@ -36,8 +36,12 @@ pub struct ChangelogEntry {
pub closes: Option<String>,
}
/// Parse the most recent entry of a Debian changelog file.
pub fn parse_changelog_entry(path: &Path) -> Result<ChangelogEntry, Box<dyn std::error::Error>> {
/// Parse up to `limit` entries of a Debian changelog file, newest first
/// (`None` parses the whole file).
pub fn parse_changelog_entries(
path: &Path,
limit: Option<usize>,
) -> Result<Vec<ChangelogEntry>, Box<dyn std::error::Error>> {
let content = std::fs::read_to_string(path).map_err(|e| {
format!(
"failed to read changelog '{}': {}. Make sure you are running \
@@ -46,17 +50,60 @@ pub fn parse_changelog_entry(path: &Path) -> Result<ChangelogEntry, Box<dyn std:
e
)
})?;
parse_changelog_entry_from_str(&content)
parse_changelog_entries_from_str(&content, limit)
}
/// Parse the most recent changelog entry from its textual content. `origin`
/// is used in error messages only.
/// Parse the most recent entry of a Debian changelog file.
pub fn parse_changelog_entry(path: &Path) -> Result<ChangelogEntry, Box<dyn std::error::Error>> {
parse_changelog_entries(path, Some(1)).map(|mut entries| entries.remove(0))
}
/// Parse the most recent changelog entry from its textual content.
pub fn parse_changelog_entry_from_str(
content: &str,
) -> Result<ChangelogEntry, Box<dyn std::error::Error>> {
parse_changelog_entries_from_str(content, Some(1)).map(|mut entries| entries.remove(0))
}
/// Parse changelog entries from their textual content, newest first.
///
/// `limit` bounds the number of parsed entries (`None` parses the whole
/// file). Content below the last entry that is not another entry header
/// (e.g. an older changelog kept in a non-Debian format) is ignored.
pub fn parse_changelog_entries_from_str(
content: &str,
limit: Option<usize>,
) -> Result<Vec<ChangelogEntry>, Box<dyn std::error::Error>> {
let origin = "changelog";
let mut lines = content.lines().peekable();
let mut entries = Vec::new();
loop {
if limit.is_some_and(|n| entries.len() >= n) {
break;
}
// Blank separators between entries.
while lines.peek().is_some_and(|l| l.trim().is_empty()) {
lines.next();
}
let Some(next) = lines.peek() else {
break;
};
if !entries.is_empty() && !looks_like_header(next.trim_end()) {
break;
}
entries.push(parse_one_entry(&mut lines, origin)?);
}
Ok(entries)
}
/// Parse one entry: header line, body, maintainer trailer. Parsing stops
/// without consuming the first line that is a trailer terminator, an emacs
/// local-variables block, or the next entry's header — the stream can then
/// be resumed for the following entry.
fn parse_one_entry(
lines: &mut std::iter::Peekable<std::str::Lines<'_>>,
origin: &str,
) -> Result<ChangelogEntry, Box<dyn std::error::Error>> {
// --- Header line: `package (version) distributions; urgency=medium[, key=value]`
let header = loop {
match lines.next() {
@@ -104,19 +151,21 @@ pub fn parse_changelog_entry_from_str(
// --- Body until trailer line ` -- Name <email> Date`
let mut body_lines: Vec<String> = Vec::new();
let mut trailer: Option<String> = None;
for line in lines {
while let Some(line) = lines.peek().copied() {
let line = line.trim_end();
if line.starts_with(" -- ") {
trailer = Some(line.to_string());
trailer = lines.next().map(|l| l.trim_end().to_string());
break;
}
// Stop at an emacs local-variables block or a new entry header.
// Stop at an emacs local-variables block or a new entry header
// (both peeked, not consumed).
if line.starts_with("Local variables:") {
break;
}
if !line.trim().is_empty() && looks_like_header(line) && !body_lines.is_empty() {
break;
}
lines.next();
// Blank lines become "." like dpkg does for the Changes field.
if line.trim().is_empty() {
body_lines.push(".".to_string());
@@ -211,39 +260,6 @@ fn find_closes(body_lines: &[String]) -> Option<String> {
)
}
/// Return the version of the *previous* changelog entry (the second header
/// in the file), or `None` when only one entry exists.
pub fn parse_previous_version(path: &Path) -> Result<Option<String>, Box<dyn std::error::Error>> {
let content = std::fs::read_to_string(path)
.map_err(|e| format!("failed to read changelog '{}': {}", path.display(), e))?;
parse_previous_version_from_str(&content)
}
/// Return the version of the *previous* changelog entry from the textual
/// content of a changelog file.
pub fn parse_previous_version_from_str(
content: &str,
) -> Result<Option<String>, Box<dyn std::error::Error>> {
let mut seen_first = false;
for line in content.lines() {
let line = line.trim_end();
if looks_like_header(line) {
if !seen_first {
seen_first = true;
continue;
}
let open = line
.find('(')
.ok_or_else(|| format!("invalid changelog header: {line}"))?;
let close = line[open..]
.find(')')
.ok_or_else(|| format!("unbalanced parenthesis in changelog header '{line}'"))?;
return Ok(Some(line[open + 1..open + close].to_string()));
}
}
Ok(None)
}
/// Heuristic check for a changelog entry header line
/// (`name (version) dist; urgency=...`).
fn looks_like_header(line: &str) -> bool {
@@ -309,4 +325,114 @@ pkg (1.0-1+b1) unstable; urgency=medium, binary-only=yes
assert!(entry.binary_only);
assert_eq!(entry.version.full(), "1.0-1+b1");
}
const THREE_ENTRIES: &str = "\
pkg (2.0-1) unstable; urgency=low
* New upstream release.
-- Pkh Tester <pkh@example.com> Thu, 01 Jan 2026 00:00:00 +0000
pkg (1.4-2) unstable; urgency=medium
* Revision bump.
-- Pkh Tester <pkh@example.com> Wed, 01 Jan 2025 00:00:00 +0000
pkg (1.4-1) unstable; urgency=medium
* Initial release.
-- Pkh Tester <pkh@example.com> Sat, 01 Mar 2025 00:00:00 +0000
";
#[test]
fn entries_parse_newest_first_with_limits() {
// Whole file.
let all = parse_changelog_entries_from_str(THREE_ENTRIES, None).unwrap();
assert_eq!(all.len(), 3);
assert_eq!(all[0].version.full(), "2.0-1");
assert_eq!(all[1].version.full(), "1.4-2");
assert_eq!(all[2].version.full(), "1.4-1");
// Bounded limits.
assert_eq!(
parse_changelog_entries_from_str(THREE_ENTRIES, Some(1))
.unwrap()
.len(),
1
);
let two = parse_changelog_entries_from_str(THREE_ENTRIES, Some(2)).unwrap();
assert_eq!(two.len(), 2);
assert_eq!(two[0].version.full(), "2.0-1");
assert_eq!(two[1].version.full(), "1.4-2");
// A limit beyond the entry count yields everything.
assert_eq!(
parse_changelog_entries_from_str(THREE_ENTRIES, Some(10))
.unwrap()
.len(),
3
);
// The single-entry helpers agree with a limit of 1.
let one = parse_changelog_entries_from_str(THREE_ENTRIES, Some(1)).unwrap();
let via_helper = parse_changelog_entry_from_str(THREE_ENTRIES).unwrap();
assert_eq!(one[0].version.full(), via_helper.version.full());
assert_eq!(one[0].source, via_helper.source);
}
#[test]
fn entries_ignore_trailing_foreign_content() {
let content = "\
pkg (1.0) unstable; urgency=medium
* Something.
-- Pkh Tester <pkh@example.com> Thu, 01 Jan 2026 00:00:00 +0000
older changelog kept in an ad-hoc format:
version 0.9 - some text, not a Debian entry
version 0.8 - more text
";
let entries = parse_changelog_entries_from_str(content, None).unwrap();
assert_eq!(entries.len(), 1);
assert_eq!(entries[0].version.full(), "1.0");
}
#[test]
fn entries_parse_body_of_later_entries() {
let entries = parse_changelog_entries_from_str(THREE_ENTRIES, Some(2)).unwrap();
// The second entry's body and trailer are fully parsed, not merely
// its header line.
assert_eq!(
entries[1].changes_field,
"\npkg (1.4-2) unstable; urgency=medium\n.\n * Revision bump."
);
assert_eq!(entries[1].maintainer_email, "pkh@example.com");
assert_eq!(entries[1].urgency, "medium");
}
#[test]
fn entries_reject_malformed_later_entry() {
let content = "\
pkg (1.0) unstable; urgency=medium
* Something.
-- Pkh Tester <pkh@example.com> Thu, 01 Jan 2026 00:00:00 +0000
pkg (0.9) unstable; urgency=medium
* No trailer below.
";
assert!(parse_changelog_entries_from_str(content, None).is_err());
// Not parsed when not requested.
assert_eq!(
parse_changelog_entries_from_str(content, Some(1))
.unwrap()
.len(),
1
);
}
}
+267 -3
View File
@@ -1,5 +1,5 @@
//! File checksum computation and formatting for `.changes` / `.buildinfo`
//! fields (MD5, SHA-1, SHA-256 + size), mirroring `Dpkg::Checksums`.
//! fields (MD5, SHA-1, SHA-256, SHA-512 + size), mirroring `Dpkg::Checksums`.
use std::collections::HashMap;
use std::io::Read;
@@ -7,7 +7,7 @@ use std::path::Path;
use md5::Md5;
use sha1::Sha1;
use sha2::{Digest, Sha256};
use sha2::{Digest, Sha256, Sha512};
/// Checksums and size of a single file.
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -20,6 +20,40 @@ pub struct Entry {
pub sha1: String,
/// Lowercase hexadecimal SHA-256 digest.
pub sha256: String,
/// Lowercase hexadecimal SHA-512 digest.
pub sha512: String,
}
/// The checksum algorithm carried by a `Checksums-*` field body, as handled
/// by [`FileChecksums::parse_field`].
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ChecksumKind {
/// SHA-1 (`Checksums-Sha1` field).
Sha1,
/// SHA-256 (`Checksums-Sha256` field).
Sha256,
/// SHA-512 (`Checksums-Sha512` field).
Sha512,
}
impl ChecksumKind {
/// The `Checksums-*` field name carrying this digest.
pub fn field_name(self) -> &'static str {
match self {
ChecksumKind::Sha1 => "Checksums-Sha1",
ChecksumKind::Sha256 => "Checksums-Sha256",
ChecksumKind::Sha512 => "Checksums-Sha512",
}
}
/// Length in lowercase hex characters of one digest of this kind.
fn digest_len(self) -> usize {
match self {
ChecksumKind::Sha1 => 40,
ChecksumKind::Sha256 => 64,
ChecksumKind::Sha512 => 128,
}
}
}
/// Compute all supported checksums of a file.
@@ -30,6 +64,7 @@ fn compute(path: &Path) -> Result<Entry, Box<dyn std::error::Error>> {
let mut md5_hasher = Md5::new();
let mut sha1_hasher = Sha1::new();
let mut sha256_hasher = Sha256::new();
let mut sha512_hasher = Sha512::new();
let mut size: u64 = 0;
let mut buf = [0u8; 64 * 1024];
@@ -41,6 +76,7 @@ fn compute(path: &Path) -> Result<Entry, Box<dyn std::error::Error>> {
md5_hasher.update(&buf[..n]);
sha1_hasher.update(&buf[..n]);
sha256_hasher.update(&buf[..n]);
sha512_hasher.update(&buf[..n]);
size += n as u64;
}
@@ -49,6 +85,7 @@ fn compute(path: &Path) -> Result<Entry, Box<dyn std::error::Error>> {
md5: hex::encode(md5_hasher.finalize()),
sha1: hex::encode(sha1_hasher.finalize()),
sha256: hex::encode(sha256_hasher.finalize()),
sha512: hex::encode(sha512_hasher.finalize()),
})
}
@@ -168,6 +205,84 @@ impl FileChecksums {
pub fn field_sha256(&self) -> String {
self.format_field(|e| &e.sha256)
}
/// Value for the `Checksums-Sha512` field, or `None` when any registered
/// file has no SHA-512 digest (e.g. entries merged from a `.dsc`, which
/// dpkg only writes with sha1/sha256 checksums): renderers omit the
/// field instead of writing an incomplete checksum list.
pub fn field_sha512(&self) -> Option<String> {
if self.iter().any(|(_, e)| e.sha512.is_empty()) {
return None;
}
Some(self.format_field(|e| &e.sha512))
}
/// Parse the body of a `Checksums-Sha1` / `Checksums-Sha256` /
/// `Checksums-Sha512` field (as rendered by [`FileChecksums::field_sha1`],
/// [`FileChecksums::field_sha256`] or [`FileChecksums::field_sha512`])
/// into `(name, entry)` pairs, ready to be fed into
/// [`FileChecksums::insert_entry`] (e.g. when consuming a `.dsc`).
///
/// Each non-blank line holds `"<hex digest> <size> <name>"`; blank lines
/// are tolerated and anything else is a malformed line, reported as an
/// error naming [`ChecksumKind::field_name`] and the offending line. Only
/// the digest selected by `kind` is filled in the returned entries: the
/// other digest fields are left empty and must be completed from the
/// remaining `Checksums-*` fields (or by recomputation) before rendering.
///
/// Note: this deliberately re-implements the line grammar of the private
/// `build::parse_checksum_field` helper (which additionally accepts the
/// legacy 5-column `Files` layout); the two are intentionally not unified
/// across modules.
pub fn parse_field(kind: ChecksumKind, value: &str) -> Result<Vec<(String, Entry)>, String> {
let mut entries = Vec::new();
for line in value.lines() {
if line.trim().is_empty() {
continue;
}
let tokens: Vec<&str> = line.split_whitespace().collect();
let [digest, size, name] = tokens.as_slice() else {
return Err(format!(
"malformed '{}' line (expected 'checksum size name', got {} \
columns): '{line}'",
kind.field_name(),
tokens.len()
));
};
let size: u64 = size.parse().map_err(|_| {
format!(
"malformed '{}' line (size '{size}' is not a number): '{line}'",
kind.field_name()
)
})?;
let digest_ok = digest.len() == kind.digest_len()
&& digest
.bytes()
.all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b));
if !digest_ok {
return Err(format!(
"malformed '{}' line (digest '{digest}' is not {} lowercase \
hex characters): '{line}'",
kind.field_name(),
kind.digest_len()
));
}
let mut entry = Entry {
size,
md5: String::new(),
sha1: String::new(),
sha256: String::new(),
sha512: String::new(),
};
match kind {
ChecksumKind::Sha1 => entry.sha1 = digest.to_string(),
ChecksumKind::Sha256 => entry.sha256 = digest.to_string(),
ChecksumKind::Sha512 => entry.sha512 = digest.to_string(),
}
entries.push((name.to_string(), entry));
}
Ok(entries)
}
}
#[cfg(test)]
@@ -184,16 +299,165 @@ mod tests {
cs.add_file(&p).unwrap();
let e = cs.get("sample.txt").unwrap();
// Verified with coreutils: echo "hello world" | md5sum / sha1sum / sha256sum
// Verified with coreutils: echo "hello world" | md5sum / sha1sum / sha256sum / sha512sum
assert_eq!(e.md5, "6f5902ac237024bdd0c176cb93063dc4");
assert_eq!(e.sha1, "22596363b3de40b06f981fb85d82312e8c0ed511");
assert_eq!(
e.sha256,
"a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447"
);
assert_eq!(
e.sha512,
"db3974a97f2407b7cae1ae637c0030687a11913274d578492558e39c16c017de\
84eacdc8c62fe34ee4e12b4b1428817f09b6a2760c3f8a664ceae94d2434a593"
);
assert_eq!(e.size, 12);
}
/// SHA-512 of the empty input is a well-known constant: a zero-size file
/// must still carry it (never an empty digest string, which is reserved
/// for "digest unknown", e.g. entries merged from a `.dsc`).
#[test]
fn sha512_of_empty_file_is_the_known_constant() {
let dir = tempfile::tempdir().unwrap();
let p = dir.path().join("empty.txt");
std::fs::write(&p, b"").unwrap();
let mut cs = FileChecksums::new();
cs.add_file(&p).unwrap();
let e = cs.get("empty.txt").unwrap();
assert_eq!(e.size, 0);
assert_eq!(
e.sha512,
"cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce\
47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e"
);
}
/// All four digests render; the `Checksums-Sha512` field round-trips
/// through [`FileChecksums::parse_field`] back into a registry with
/// identical names (insertion order), sizes and SHA-512 digests.
#[test]
fn sha512_field_round_trip() {
let dir = tempfile::tempdir().unwrap();
let a = dir.path().join("a.txt");
let b = dir.path().join("b.txt");
std::fs::write(&a, b"aaa").unwrap();
std::fs::write(&b, b"bb").unwrap();
let mut cs = FileChecksums::new();
cs.add_file(&b).unwrap();
cs.add_file(&a).unwrap();
// Every digest kind must be populated and render a full field.
assert!(!cs.field_md5().is_empty());
assert!(!cs.field_sha1().is_empty());
assert!(!cs.field_sha256().is_empty());
let sha512_field = cs.field_sha512().expect("all entries have sha512");
// Round-trip the Checksums-Sha512 field through the parser.
let mut reparsed = FileChecksums::new();
for (key, entry) in FileChecksums::parse_field(ChecksumKind::Sha512, &sha512_field).unwrap()
{
reparsed.insert_entry(&key, entry);
}
let keys: Vec<&str> = reparsed.iter().map(|(k, _)| k.as_str()).collect();
assert_eq!(keys, vec!["b.txt", "a.txt"], "insertion order preserved");
for (key, e) in cs.iter() {
let got = reparsed.get(key).unwrap();
assert_eq!(got.size, e.size, "{key}");
assert_eq!(got.sha512, e.sha512, "{key}");
}
// Rendering the re-parsed registry yields the same field value.
assert_eq!(reparsed.field_sha512().unwrap(), sha512_field);
// The parser dispatches on `kind`: a Checksums-Sha256 body fills the
// sha256 column, leaving the others (including sha512) unknown.
let (key, entry) =
&FileChecksums::parse_field(ChecksumKind::Sha256, &cs.field_sha256()).unwrap()[0];
assert_eq!(entry.sha256, cs.get(key).unwrap().sha256);
assert!(entry.sha512.is_empty());
let (_, entry) =
&FileChecksums::parse_field(ChecksumKind::Sha512, &sha512_field).unwrap()[0];
assert!(entry.md5.is_empty() && entry.sha1.is_empty() && entry.sha256.is_empty());
assert!(!entry.sha512.is_empty());
}
/// Malformed `Checksums-Sha512` bodies (wrong column count, non-numeric
/// size, wrong digest shape) must be rejected with an error naming the
/// field and the offending line; blank lines are tolerated.
#[test]
fn parse_field_rejects_malformed_lines() {
// 128 lowercase hex characters, as rendered by field_sha512.
let digest = "ab".repeat(64);
// Blank lines are skipped.
let entries =
FileChecksums::parse_field(ChecksumKind::Sha512, &format!("\n {digest} 12 a.txt\n\n"))
.unwrap();
assert_eq!(entries.len(), 1);
assert_eq!(entries[0].0, "a.txt");
assert_eq!(entries[0].1.size, 12);
// 2 columns: missing the name.
let err =
FileChecksums::parse_field(ChecksumKind::Sha512, &format!("{digest} 12")).unwrap_err();
assert!(err.contains("Checksums-Sha512"), "{err}");
assert!(err.contains(&format!("{digest} 12")), "{err}");
// 4 columns.
let err =
FileChecksums::parse_field(ChecksumKind::Sha512, &format!(" {digest} 12 bogus a.txt"))
.unwrap_err();
assert!(err.contains("Checksums-Sha512"), "{err}");
assert!(err.contains("a.txt"), "{err}");
// Non-numeric size.
let err =
FileChecksums::parse_field(ChecksumKind::Sha512, &format!(" {digest} twelve a.txt"))
.unwrap_err();
assert!(err.contains("not a number"), "{err}");
assert!(err.contains("twelve"), "{err}");
// Digest that is not 128 lowercase hex characters.
let err = FileChecksums::parse_field(ChecksumKind::Sha512, " abc123 12 a.txt").unwrap_err();
assert!(err.contains("lowercase hex"), "{err}");
}
/// `field_sha512` is only-if-populated: an entry merged without a SHA-512
/// digest (e.g. taken from a `.dsc`) suppresses the whole field instead
/// of rendering an incomplete checksum list.
#[test]
fn field_sha512_omitted_when_any_digest_missing() {
let dir = tempfile::tempdir().unwrap();
let a = dir.path().join("a.txt");
std::fs::write(&a, b"aaa").unwrap();
let mut cs = FileChecksums::new();
cs.add_file(&a).unwrap();
assert!(cs.field_sha512().is_some());
cs.insert_entry(
"from.dsc",
Entry {
size: 12,
md5: "d41d8cd98f00b204e9800998ecf8427e".to_string(),
sha1: "da39a3ee5e6b4b0d3255bfef95601890afd80709".to_string(),
sha256: format!("e3b0{:0>62}", "0"),
sha512: String::new(), // not recorded in .dsc files
},
);
assert!(
cs.field_sha512().is_none(),
"one incomplete entry must suppress Checksums-Sha512"
);
// The other kinds are unaffected.
assert!(!cs.field_md5().is_empty());
assert!(!cs.field_sha1().is_empty());
assert!(!cs.field_sha256().is_empty());
}
#[test]
fn insertion_order_preserved() {
let dir = tempfile::tempdir().unwrap();
+186 -8
View File
@@ -10,9 +10,14 @@ use std::path::Path;
/// A single deb822 paragraph: an ordered list of `(field, value)` pairs.
///
/// The parser is lenient: duplicate field names are kept as separate entries
/// (accessors see the first one; `set` collapses them back to a single one).
///
/// Values are stored with continuation-line breaks as `\n` and without the
/// leading whitespace of continuation lines. Serialization re-adds a single
/// leading space in front of every continuation line, matching dpkg output.
/// leading space in front of every continuation line, matching dpkg output;
/// blank lines inside a value are encoded as ` .` (and decoded back) so they
/// survive a write/parse round-trip.
#[derive(Debug, Clone, Default, PartialEq)]
pub struct Paragraph {
fields: Vec<(String, String)>,
@@ -25,6 +30,10 @@ impl Paragraph {
}
/// Look up a field value (case-insensitive field name).
///
/// Returns the first match. The parser is lenient and keeps duplicate
/// field names as-is; use [`Paragraph::iter`] to reach the other
/// occurrences. [`Paragraph::set`] collapses them.
pub fn get(&self, field: &str) -> Option<&str> {
self.fields
.iter()
@@ -32,17 +41,26 @@ impl Paragraph {
.map(|(_, v)| v.as_str())
}
/// Set a field value, replacing any previous occurrence (case-insensitive).
/// Appends the field at the end if it did not exist yet.
/// Set a field value, replacing all case-insensitive duplicates: after
/// the call at most one entry with this field name remains — the updated
/// one, kept at its original position. Appends the field at the end if
/// no entry existed yet.
pub fn set(&mut self, field: &str, value: &str) {
for (k, v) in self.fields.iter_mut() {
let mut updated = false;
self.fields.retain_mut(|(k, v)| {
if k.eq_ignore_ascii_case(field) {
if updated {
return false;
}
*v = value.to_string();
return;
}
updated = true;
}
true
});
if !updated {
self.fields.push((field.to_string(), value.to_string()));
}
}
/// Remove a field (case-insensitive). Returns true if it was present.
pub fn remove(&mut self, field: &str) -> bool {
@@ -66,7 +84,9 @@ impl Paragraph {
///
/// Comment lines (starting with `#`) are ignored. Blank lines separate
/// paragraphs. Continuation lines must start with a space or a tab; exactly
/// one leading space (or tab) is stripped from the stored value.
/// one leading space (or tab) is stripped from the stored value, and a
/// continuation whose content is a lone `.` decodes to an empty line
/// (dpkg's encoding for blank lines inside field values).
pub fn parse_paragraphs(input: &str) -> Vec<Paragraph> {
let mut paragraphs = Vec::new();
let mut current = Paragraph::new();
@@ -89,7 +109,14 @@ pub fn parse_paragraphs(input: &str) -> Vec<Paragraph> {
// Continuation line
if line.starts_with(' ') || line.starts_with('\t') {
let content = line.strip_prefix(' ').unwrap_or(line);
// Exactly one leading space or tab is stripped.
let content = line
.strip_prefix(' ')
.or_else(|| line.strip_prefix('\t'))
.unwrap_or(line);
// dpkg encodes a blank line inside a value as a lone `.` after
// the leading whitespace; mirror that on read.
let content = if content == "." { "" } else { content };
if let Some(field) = &last_field
&& let Some((_, v)) = current
.fields
@@ -147,14 +174,65 @@ pub fn write_paragraph(p: &Paragraph) -> String {
}
for line in lines {
out.push('\n');
if line.is_empty() {
// dpkg encodes a blank line inside a value as ` .`; writing a
// bare continuation line would be mistaken for a paragraph
// separator on re-parse and silently drop the rest.
out.push_str(" .");
} else {
out.push(' ');
out.push_str(line);
}
}
out.push('\n');
}
out
}
/// Return the signed body of a clearsigned message, as a slice of `text`.
///
/// If `text` starts with the OpenPGP clearsigned-marker line, the armor
/// header block (the `Hash: ...` line and any `Comment:` lines, up to and
/// including the blank line that closes the header) is skipped, and the
/// result is cut at the `-----BEGIN PGP SIGNATURE-----` marker so the
/// signature trailer is dropped as well. This keeps the armor metadata from
/// being parsed as deb822 fields (`Hash:` would otherwise land in the first
/// stanza and `Comment:` in the last one).
///
/// Input that is not clearsigned is returned unchanged, so callers can apply
/// this unconditionally before parsing.
pub fn strip_clearsigned_armour(text: &str) -> &str {
const BEGIN_SIGNED: &str = "-----BEGIN PGP SIGNED MESSAGE-----";
const BEGIN_SIGNATURE: &str = "-----BEGIN PGP SIGNATURE-----";
if !text.starts_with(BEGIN_SIGNED) {
return text;
}
// Walk past the armor headers to the blank line that precedes the body.
let mut body = text;
loop {
match body.split_once('\n') {
Some((line, remainder)) => {
body = remainder;
// An empty line ends the armor header block (`\r` covers a
// CRLF-terminated blank line).
if line.is_empty() || line == "\r" {
break;
}
}
// Malformed armor: no body at all.
None => return "",
}
}
// Cut off the signature block, if present.
match body.find(BEGIN_SIGNATURE) {
Some(i) => &body[..i],
None => body,
}
}
#[cfg(test)]
mod tests {
use super::*;
@@ -197,6 +275,75 @@ mod tests {
assert_eq!(reparsed[0].get("Description"), Some(value));
}
#[test]
fn blank_lines_survive_roundtrip() {
let mut p = Paragraph::new();
p.set("Description", "a\n\nb");
let text = write_paragraph(&p);
// dpkg encoding: a blank line inside a value is written as ` .`.
assert_eq!(text, "Description: a\n .\n b\n");
// parse -> write -> parse must not lose data.
let reparsed = parse_paragraphs(&text);
assert_eq!(reparsed[0].get("Description"), Some("a\n\nb"));
assert_eq!(
parse_paragraphs(&write_paragraph(&reparsed[0]))[0].get("Description"),
Some("a\n\nb")
);
}
#[test]
fn parse_lone_dot_continuation_is_blank_line() {
let paras = parse_paragraphs("Description:\n a\n .\n b\n");
assert_eq!(paras[0].get("Description"), Some("\na\n\nb"));
}
#[test]
fn tab_continuation_strips_exactly_one_tab() {
let paras = parse_paragraphs("Description: a\n\tb\n\t\tdeep\n");
assert_eq!(paras[0].get("Description"), Some("a\nb\n\tdeep"));
}
#[test]
fn strip_armour_extracts_signed_dsc_body() {
let signed = "\
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Format: 3.0 (native)
Source: hello
Binary: hello
Architecture: any
Version: 1.0-1
Checksums-Sha256:
abc 100 hello_1.0.tar.gz
-----BEGIN PGP SIGNATURE-----
iQEcBAABCgAGBQJabcdAAoJEL abc
-----END PGP SIGNATURE-----
";
let body = strip_clearsigned_armour(signed);
assert!(body.starts_with("Format:"));
assert!(!body.contains("SIGNATURE"));
let paras = parse_paragraphs(body);
assert_eq!(paras.len(), 1);
// The armor `Hash:` header must not land in the stanza...
assert!(paras[0].get("Hash").is_none());
assert_eq!(paras[0].get("Source"), Some("hello"));
// ...and the signature trailer must not contribute a `Comment:` field.
assert!(paras[0].get("Comment").is_none());
assert_eq!(
paras[0].get("Checksums-Sha256"),
Some("\nabc 100 hello_1.0.tar.gz")
);
}
#[test]
fn strip_armour_passes_unsigned_text_through() {
let plain = "Source: hello\nVersion: 1.0\n";
assert_eq!(strip_clearsigned_armour(plain), plain);
}
#[test]
fn set_replaces_case_insensitive() {
let mut p = Paragraph::new();
@@ -206,6 +353,37 @@ mod tests {
assert_eq!(p.iter().count(), 1);
}
#[test]
fn lenient_parse_keeps_duplicate_fields() {
let paras = parse_paragraphs("Package: hello\nDepends: a\ndepends: b\n");
let p = &paras[0];
// deb822 forbids duplicate fields but the parser is lenient and keeps
// both entries; `get` returns the first.
let depends: Vec<_> = p
.iter()
.filter(|(k, _)| k.eq_ignore_ascii_case("Depends"))
.collect();
assert_eq!(depends, [("Depends", "a"), ("depends", "b")]);
assert_eq!(p.get("Depends"), Some("a"));
}
#[test]
fn set_removes_case_insensitive_duplicates() {
let mut paras = parse_paragraphs("Package: hello\nDepends: a\ndepends: b\n");
let mut p = paras.remove(0);
p.set("Depends", "c");
// Exactly one depends-family entry remains, with the new value.
let depends: Vec<_> = p
.iter()
.filter(|(k, _)| k.eq_ignore_ascii_case("Depends"))
.collect();
assert_eq!(depends, [("Depends", "c")]);
assert_eq!(p.get("depends"), Some("c"));
// ...kept at its original position, and a write round-trip no longer
// leaks the stale duplicate.
assert_eq!(write_paragraph(&p), "Package: hello\nDepends: c\n");
}
#[test]
fn remove_field() {
let mut p = Paragraph::new();
+506 -25
View File
@@ -10,7 +10,7 @@
//! (<https://manpages.debian.org/libdpkg-perl>) and were validated
//! differentially against the real tool.
use std::collections::HashMap;
use std::collections::{HashMap, HashSet};
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
@@ -187,12 +187,16 @@ pub fn parse_simple(dep: &str, build_dep: bool) -> Result<PkgRelation, String> {
let constraint = match (caps.get(5), caps.get(6)) {
(Some(op), Some(version)) => {
// The deprecated single-character spellings `<` and `>` were
// "confusingly defined" (Debian Policy §7) to mean earlier-or-equal
// and later-or-equal, i.e. `<=` and `>=`; dpkg still accepts them
// with those non-strict semantics.
let relation = match op.as_str() {
"<<" | "<" => Relation::Lt,
"<=" => Relation::Le,
"<<" => Relation::Lt,
"<" | "<=" => Relation::Le,
"=" => Relation::Eq,
">=" => Relation::Ge,
">>" | ">" => Relation::Gt,
">" | ">=" => Relation::Ge,
">>" => Relation::Gt,
other => return Err(format!("invalid relation '{other}' in '{dep}'")),
};
let version = DebianVersion::parse(version.as_str())
@@ -579,6 +583,24 @@ pub struct Facts {
build_arch: String,
installed: HashMap<String, Vec<InstalledPkg>>,
provided: HashMap<String, Vec<ProvidedPkg>>,
/// Virtual packages whose `Provides` entries had to be rejected as a
/// whole (unparseable field, malformed version, or a relation other
/// than `=`), mirroring a dpkg rejection of the entry: versioned
/// relations on them stay undecidable instead of turning unmet.
unreadable_provides: HashSet<String>,
}
/// Best-effort virtual package names mentioned in a raw `Provides` value,
/// used to remember rejected entries whose content cannot even be parsed
/// (the full dependency grammar is deliberately not re-applied here).
fn raw_provides_names(field: &str) -> impl Iterator<Item = &str> {
field
.split([',', '|'])
.filter_map(|alt| alt.split_whitespace().next())
.map(|tok| match tok.split_once(':') {
Some((name, _qual)) => name,
None => tok,
})
}
impl Facts {
@@ -589,6 +611,7 @@ impl Facts {
build_arch: build_arch.to_string(),
installed: HashMap::new(),
provided: HashMap::new(),
unreadable_provides: HashSet::new(),
}
}
@@ -605,6 +628,11 @@ impl Facts {
}
/// Record that `provider` provides the virtual package `virtual_name`.
///
/// A `relation` other than [`Relation::Eq`], or a `version` that fails
/// to parse, makes the provide unusable: versioned relations on
/// `virtual_name` then evaluate to `None` (undecidable) instead of a
/// possibly-wrong `Some(false)`.
pub fn add_provided(
&mut self,
virtual_name: &str,
@@ -635,7 +663,10 @@ impl Facts {
/// Only stanzas whose `Status` ends with `ok installed` participate;
/// their `Provides` field registers versioned/unversioned virtual
/// packages (architecture-restricted provides are reduced against
/// `host_arch`).
/// `host_arch`). A `Provides` field that fails to parse, or that
/// carries a relation other than `=`, is rejected as a whole (like
/// dpkg rejects the entry) and its virtual names are remembered as
/// unreadable.
pub fn from_status(content: &str, host_arch: &str, build_arch: &str) -> Facts {
let mut facts = Facts::new(host_arch, build_arch);
for para in crate::debian::control::parse_paragraphs(content) {
@@ -660,20 +691,38 @@ impl Facts {
union: true,
build_dep: false,
};
// Virtual (Provides) fields only accept '=' relations; a
// parse failure skips the whole field, like dpkg does.
let Ok(parsed) = Deps::parse_inner(provides, &opts, true) else {
// Virtual (Provides) fields only accept versionless or
// exactly '='-versioned alternatives; a field that fails
// to parse, or that carries any other relation, is
// rejected as a whole, like dpkg rejects the entry. The
// mentioned virtual names are remembered so versioned
// relations on them stay undecidable instead of turning
// into a possibly-wrong "unmet" verdict.
let mut parsed = None;
let mut rejected: Vec<String> = Vec::new();
match Deps::parse_inner(provides, &opts, true) {
Ok(deps) => {
if deps.clauses().flatten().any(|alt| {
alt.constraint
.as_ref()
.is_some_and(|c| c.relation != Relation::Eq)
}) {
rejected
.extend(deps.clauses().flatten().map(|alt| alt.package.clone()));
} else {
parsed = Some(deps);
}
}
// The field could not even be parsed: recover the raw
// names so the rejection is still remembered.
Err(_) => rejected.extend(raw_provides_names(provides).map(str::to_string)),
}
facts.unreadable_provides.extend(rejected);
let Some(parsed) = parsed else {
continue;
};
for clause in parsed.clauses() {
for alt in clause {
if alt
.constraint
.as_ref()
.is_some_and(|c| c.relation != Relation::Eq)
{
continue;
}
let provided_version = alt
.constraint
.as_ref()
@@ -751,11 +800,21 @@ impl Facts {
}
}
// A rejected Provides entry for this virtual package carries
// information that could not be read: a versioned relation can
// then not be decided (an unversioned one only needs the name,
// which the rejection removed).
if rel.constraint.is_some() && self.unreadable_provides.contains(&rel.package) {
lackinfos = true;
}
if let Some(providers) = self.provided.get(&rel.package) {
for vp in providers {
// Only unversioned provides and strictly-versioned provides
// can satisfy a dependency.
// Only unversioned provides and exactly-versioned provides
// can satisfy a dependency; anything else is an invalid
// provide and leaves the relation undecidable.
if vp.relation.is_some_and(|r| r != Relation::Eq) {
lackinfos = true;
continue;
}
match &rel.constraint {
@@ -763,11 +822,16 @@ impl Facts {
let Some(vp_version) = &vp.version else {
continue;
};
if let Ok(vp_v) = DebianVersion::parse(vp_version)
&& constraint.relation.eval(&vp_v, &constraint.version)
{
match DebianVersion::parse(vp_version) {
Ok(vp_v) if constraint.relation.eval(&vp_v, &constraint.version) => {
return Some(true);
}
// An unreadable provided version, like an
// unreadable installed version, makes the
// relation undecidable instead of unmet.
Err(_) => lackinfos = true,
Ok(_) => {}
}
}
None => return Some(true),
}
@@ -781,8 +845,15 @@ impl Facts {
/// Options for [`check_build_depends`].
#[derive(Debug, Clone)]
pub struct CheckOpts {
/// Host architecture (defaults to the native architecture).
/// Host architecture, i.e. `DEB_HOST_ARCH`: the architecture the
/// packages are built FOR (defaults to the native architecture).
/// Bracketed `foo [arch]` restrictions evaluate against it.
pub host_arch: String,
/// Build architecture, i.e. `DEB_BUILD_ARCH`: the architecture the
/// build runs ON (defaults to the native architecture). `:native`
/// dependency qualifiers and the dpkg status attribution of the
/// build-side facts resolve against it.
pub build_arch: String,
/// Active build profiles.
pub build_profiles: Vec<String>,
/// Ignore `Build-Depends-Arch`/`Build-Conflicts-Arch` (`-A`).
@@ -800,6 +871,7 @@ impl Default for CheckOpts {
fn default() -> Self {
CheckOpts {
host_arch: arch::native().unwrap_or_default(),
build_arch: arch::native().unwrap_or_default(),
build_profiles: Vec::new(),
ignore_arch: false,
ignore_indep: false,
@@ -900,14 +972,14 @@ pub fn check_build_depends(control: &ControlInfo, opts: &CheckOpts) -> Result<Un
let bc_value = bc_parts.join(", ");
let status_path = opts.admindir.join("status");
let facts = Facts::load_status(&status_path, &opts.host_arch, &opts.host_arch)?;
let facts = Facts::load_status(&status_path, &opts.host_arch, &opts.build_arch)?;
let mut report = UnmetReport::default();
if !bd_value.trim().is_empty() {
let parse_opts = ParseOpts {
host_arch: opts.host_arch.clone(),
build_arch: opts.host_arch.clone(),
build_arch: opts.build_arch.clone(),
build_profiles: opts.build_profiles.clone(),
reduce_restrictions: true,
union: false,
@@ -929,7 +1001,7 @@ pub fn check_build_depends(control: &ControlInfo, opts: &CheckOpts) -> Result<Un
if !bc_value.trim().is_empty() {
let parse_opts = ParseOpts {
host_arch: opts.host_arch.clone(),
build_arch: opts.host_arch.clone(),
build_arch: opts.build_arch.clone(),
build_profiles: opts.build_profiles.clone(),
reduce_restrictions: true,
union: true,
@@ -1024,6 +1096,56 @@ mod tests {
assert!(Deps::parse("foo:native", &opts("amd64", &[])).is_ok());
}
/// The deprecated single-character operators `<` and `>` mean `<=` and
/// `>=` (Debian Policy §7, dpkg behavior), and canonicalize on output.
#[test]
fn legacy_single_char_relations() {
let rel = |dep: &str| {
parse_simple(dep, true)
.unwrap()
.constraint
.unwrap()
.relation
};
assert_eq!(rel("foo (<< 1.0)"), Relation::Lt);
assert_eq!(rel("foo (< 1.0)"), Relation::Le);
assert_eq!(rel("foo (<= 1.0)"), Relation::Le);
assert_eq!(rel("foo (= 1.0)"), Relation::Eq);
assert_eq!(rel("foo (>= 1.0)"), Relation::Ge);
assert_eq!(rel("foo (> 1.0)"), Relation::Ge);
assert_eq!(rel("foo (>> 1.0)"), Relation::Gt);
// Rendering always uses the canonical modern spellings.
for (dep, rendered) in [
("foo (< 1.0)", "foo (<= 1.0)"),
("foo (> 1.0)", "foo (>= 1.0)"),
("foo (<< 1.0)", "foo (<< 1.0)"),
("foo (>> 1.0)", "foo (>> 1.0)"),
] {
assert_eq!(parse_simple(dep, true).unwrap().output(), rendered);
}
}
/// `foo (< 1.0)` is satisfied by an installed `foo 1.0` (the legacy
/// operator is non-strict); `foo (< 0.9)` is not.
#[test]
fn legacy_single_char_evaluation() {
let mut facts = Facts::new("amd64", "amd64");
facts.add_installed("foo", "1.0", "amd64", "no");
let o = |s: &str| parse_simple(s, true).unwrap();
assert_eq!(facts.evaluate_relation(&o("foo (< 1.0)")), Some(true));
assert_eq!(facts.evaluate_relation(&o("foo (<= 1.0)")), Some(true));
assert_eq!(facts.evaluate_relation(&o("foo (< 0.9)")), Some(false));
assert_eq!(facts.evaluate_relation(&o("foo (<< 1.0)")), Some(false));
assert_eq!(facts.evaluate_relation(&o("foo (> 1.0)")), Some(true));
assert_eq!(facts.evaluate_relation(&o("foo (>= 1.0)")), Some(true));
assert_eq!(facts.evaluate_relation(&o("foo (> 1.1)")), Some(false));
assert_eq!(facts.evaluate_relation(&o("foo (>> 1.0)")), Some(false));
}
/// Ported from dpkg `t/Dpkg_Deps.t`: architecture reduction.
#[test]
fn arch_reduction() {
@@ -1189,6 +1311,300 @@ Provides: old-virtual (= 0.5)
assert_eq!(facts.evaluate_relation(&o("old-virtual")), Some(true));
}
/// A `Provides` entry that dpkg would reject as a whole (a malformed
/// provided version, or a relation other than `=`) must not degrade to
/// a bogus "unmet" verdict: versioned relations on the affected virtual
/// names stay undecidable.
#[test]
fn corrupt_provides_are_undecidable_not_unmet() {
// `not-a-version!` is rejected by DebianVersion::parse ('!' is not
// a legal version character), unlike dpkg-invalid but here-valid
// letter-only spellings.
let status = "\
Package: bad-version-provider
Status: install ok installed
Version: 1.0
Architecture: amd64
Provides: virt (= not-a-version!)
Package: bad-relation-provider
Status: install ok installed
Version: 1.0
Architecture: amd64
Provides: virt2 (>= 1.0), plain
";
let facts = Facts::from_status(status, "amd64", "amd64");
let o = |s: &str| parse_simple(s, true).unwrap();
// The provided version fails to parse: undecidable, not unmet.
assert_eq!(facts.evaluate_relation(&o("virt (>= 0.5)")), None);
// A non-'=' relation invalidates the whole Provides field, so
// neither of its alternatives may produce a verdict.
assert_eq!(facts.evaluate_relation(&o("virt2 (>= 0.5)")), None);
assert_eq!(facts.evaluate_relation(&o("plain (>= 0.5)")), None);
// Unversioned relations only need the name, which the rejected
// entries no longer provide: genuinely unmet.
assert_eq!(facts.evaluate_relation(&o("virt")), Some(false));
assert_eq!(facts.evaluate_relation(&o("virt2")), Some(false));
assert_eq!(facts.evaluate_relation(&o("plain")), Some(false));
}
/// Cross-compilation semantics of the package lookup (the `Facts`
/// host/build split), mirroring `Dpkg::Deps::KnownFacts::_find_package`:
/// an unqualified dependency is satisfied by the HOST architecture
/// instance, by any instance of a `Multi-Arch: foreign` package, or by
/// an `Architecture: all` instance — never by a foreign-arch instance
/// of a `Multi-Arch: no`/`same` package. The build-architecture
/// instances only come into play through `:native`.
///
/// Verdicts marked «live» were probed against `dpkg-checkbuilddeps -a
/// <host>` on a real amd64 system carrying the build-arch instances.
#[test]
fn cross_lookup_matrix() {
const O: fn(&str) -> PkgRelation = |s| parse_simple(s, true).unwrap();
// host = arm64 (target), build = amd64 (machine): the instances
// below simulate what a cross-building amd64 machine has installed.
let facts = |ma_build: &str, ma_host: &str| {
let mut f = Facts::new("arm64", "amd64");
if !ma_build.is_empty() {
f.add_installed("t", "1.0", "amd64", ma_build);
}
if !ma_host.is_empty() {
f.add_installed("t", "1.0", "arm64", ma_host);
}
f
};
// Unqualified: only the host-arch instance satisfies...
assert_eq!(
facts("no", "").evaluate_relation(&O("t")),
Some(false),
"M-A:no build-arch instance must not satisfy an unqualified dep"
);
assert_eq!(
facts("same", "").evaluate_relation(&O("t")),
Some(false),
"M-A:same build-arch instance must not satisfy an unqualified dep"
);
assert_eq!(facts("", "no").evaluate_relation(&O("t")), Some(true));
assert_eq!(facts("", "same").evaluate_relation(&O("t")), Some(true));
// ...unless the package is Multi-Arch: foreign («live»: bison,
// flex: the natively-installed variant satisfies the cross check).
assert_eq!(facts("foreign", "").evaluate_relation(&O("t")), Some(true));
assert_eq!(facts("", "foreign").evaluate_relation(&O("t")), Some(true));
// `Architecture: all` instances satisfy unqualified dependencies
// whatever the Multi-Arch attribute.
let mut all = Facts::new("arm64", "amd64");
all.add_installed("t", "1.0", "all", "foreign");
assert_eq!(all.evaluate_relation(&O("t")), Some(true));
let mut all2 = Facts::new("arm64", "amd64");
all2.add_installed("t", "1.0", "all", "no");
assert_eq!(all2.evaluate_relation(&O("t")), Some(true));
// Versioned relations check the first matching instance only:
// insertion order decides which instance a dependency binds to,
// and an unsatisfying version does not fall through to later
// instances.
let mut mixed = Facts::new("arm64", "amd64");
mixed.add_installed("t", "0.5", "arm64", "no");
mixed.add_installed("t", "3.0", "amd64", "no");
assert_eq!(mixed.evaluate_relation(&O("t (>= 1)")), Some(false));
assert_eq!(mixed.evaluate_relation(&O("t (<< 1)")), Some(true));
// `:native`: the build-architecture instance satisfies («live»:
// gcc:native on an amd64 machine, whatever the target); an
// Architecture: all instance does too — but a Multi-Arch: foreign
// instance aborts the whole lookup, even on the build architecture
// («live»: flex:native with natively-installed M-A:foreign flex is
// unmet).
assert_eq!(
facts("no", "").evaluate_relation(&O("t:native")),
Some(true)
);
assert_eq!(
facts("same", "").evaluate_relation(&O("t:native")),
Some(true)
);
assert_eq!(
facts("foreign", "").evaluate_relation(&O("t:native")),
Some(false)
);
// An Architecture: all instance satisfies :native — unless it is
// Multi-Arch: foreign, which aborts the lookup like any foreign
// instance.
assert_eq!(all2.evaluate_relation(&O("t:native")), Some(true));
assert_eq!(all.evaluate_relation(&O("t:native")), Some(false));
assert_eq!(
facts("", "no").evaluate_relation(&O("t:native")),
Some(false)
);
// `:any`: only a Multi-Arch: allowed instance satisfies, on any
// architecture («live»: libssl-dev:any with M-A:same libssl-dev is
// unmet).
assert_eq!(
facts("allowed", "").evaluate_relation(&O("t:any")),
Some(true)
);
assert_eq!(
facts("", "allowed").evaluate_relation(&O("t:any")),
Some(true)
);
assert_eq!(
facts("same", "").evaluate_relation(&O("t:any")),
Some(false)
);
// Explicit architecture qualifier: only that exact instance.
assert_eq!(facts("no", "").evaluate_relation(&O("t:amd64")), Some(true));
assert_eq!(
facts("", "no").evaluate_relation(&O("t:amd64")),
Some(false)
);
}
/// The same cross matrix, validated against the real
/// `dpkg-checkbuilddeps`: for each fixture the exit status and the
/// reported unmet list must match, with host != build (the machine is
/// the native architecture; the host architecture is a foreign one).
#[test]
fn diff_checkbuilddeps_cross_matrix() {
let build_arch = arch::native().unwrap_or_else(|_| "amd64".into());
// Any foreign arch the dpkg tables know; the instances only exist
// in the synthetic status file.
let host_arch = if build_arch == "arm64" {
"riscv64".to_string()
} else {
"arm64".to_string()
};
let mk_status = |entries: &[(&str, &str)]| {
let mut s = String::new();
for (pkg_arch, ma) in entries {
let ma = if ma.is_empty() { "no" } else { ma };
s.push_str(&format!(
"Package: t\nStatus: install ok installed\nVersion: 1.0\nArchitecture: {pkg_arch}\nMulti-Arch: {ma}\n\n"
));
}
s
};
for (name, entries, dep) in [
// Unqualified: build-arch instances never satisfy, host-arch
// and Multi-Arch: foreign do.
("ma-no-build", &[("amd64", "no")] as &[(&str, &str)], "t"),
("ma-same-build", &[("amd64", "same")], "t"),
("ma-no-host", &[("arm64", "no")], "t"),
("ma-same-host", &[("arm64", "same")], "t"),
("ma-foreign-build", &[("amd64", "foreign")], "t"),
("all-build", &[("all", "foreign")], "t"),
// :native and :any qualifiers.
("native-build", &[("amd64", "no")], "t:native"),
("native-foreign-build", &[("amd64", "foreign")], "t:native"),
("any-allowed-build", &[("amd64", "allowed")], "t:any"),
("any-same-build", &[("amd64", "same")], "t:any"),
("explicit-build", &[("amd64", "no")], "t:amd64"),
("explicit-host", &[("arm64", "no")], "t:amd64"),
] {
// Substitute the foreign architecture for fixtures that name
// the host arch explicitly.
let dep = dep.replace("arm64", &host_arch);
let entries: Vec<(String, &str)> = entries
.iter()
.map(|(a, m)| (a.replace("amd64", &build_arch), *m))
.collect();
let entries: Vec<(&str, &str)> =
entries.iter().map(|(a, m)| (a.as_str(), *m)).collect();
let status = mk_status(&entries);
let verdict = |bd: &str, host: &str| {
diff_cross_case(bd, &status, &build_arch, host, |ours, real| {
assert_eq!(ours, real, "verdict mismatch for {name}")
})
};
verdict(&dep, &host_arch);
}
}
/// One differential cross case: run the real `dpkg-checkbuilddeps`
/// with `-a <host>` against a synthetic admindir, and the native
/// checker with the equivalent options on the same control, then hand
/// both verdicts to `compare`.
fn diff_cross_case(
bd: &str,
status: &str,
build_arch: &str,
host_arch: &str,
compare: impl Fn(bool, bool),
) {
let control_text = format!(
"Source: t\nMaintainer: a <a@b.c>\nBuild-Depends: {bd}\n\nPackage: t\nArchitecture: any\nDescription: x\n y\n"
);
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("control"), &control_text).unwrap();
let admindir = dir.path().join("admin");
std::fs::create_dir_all(&admindir).unwrap();
std::fs::write(admindir.join("status"), status).unwrap();
let output = std::process::Command::new("dpkg-checkbuilddeps")
.current_dir(dir.path())
.env("LC_ALL", "C")
.arg("--admindir")
.arg(&admindir)
.arg("-a")
.arg(host_arch)
.arg("-I")
.arg("control")
.output()
.expect("run dpkg-checkbuilddeps (is dpkg-dev installed?)");
let real_ok = output.status.success();
let opts = CheckOpts {
host_arch: host_arch.to_string(),
build_arch: build_arch.to_string(),
build_profiles: Vec::new(),
ignore_arch: false,
ignore_indep: false,
ignore_builtin: true,
admindir: admindir.to_path_buf(),
};
let control = ControlInfo::parse_content(&control_text).unwrap();
let ours_ok = check_build_depends(&control, &opts)
.expect("native check failure")
.is_ok();
compare(ours_ok, real_ok);
}
/// The same undecidable verdicts through the direct facts API: an
/// unreadable provided version and an invalid (non-`=`) provide each
/// leave a versioned relation undecided, while a readable provider
/// elsewhere still satisfies it.
#[test]
fn unreadable_provided_version_and_relation_are_undecidable() {
let o = |s: &str| parse_simple(s, true).unwrap();
let mut facts = Facts::new("amd64", "amd64");
facts.add_installed("provider", "1.0", "amd64", "no");
// `virt (= not-a-version!)`: the version string fails to parse.
facts.add_provided(
"virt",
Some(Relation::Eq),
Some("not-a-version!"),
"provider",
);
assert_eq!(facts.evaluate_relation(&o("virt (>= 0.5)")), None);
// `virt2 (>= 1.0)`: a non-'=' provide is invalid.
facts.add_provided("virt2", Some(Relation::Ge), Some("1.0"), "provider");
assert_eq!(facts.evaluate_relation(&o("virt2 (>= 0.5)")), None);
// A readable provider decides the relation when it matches;
// otherwise the unreadable entry keeps it undecidable.
facts.add_provided("virt", Some(Relation::Eq), Some("2.0"), "better");
assert_eq!(facts.evaluate_relation(&o("virt (>= 0.5)")), Some(true));
assert_eq!(facts.evaluate_relation(&o("virt (>> 2.0)")), None);
}
#[test]
fn simplify_reports_unmet() {
let facts = Facts::from_status(STATUS, "amd64", "amd64");
@@ -1309,4 +1725,69 @@ Architecture: amd64
let report = check_build_depends(&control_conflict, &opts).unwrap();
assert_eq!(report.message(), "unmet build conflicts: mypackage");
}
#[test]
fn check_build_depends_cross_native_qualifier() {
let dir = tempfile::tempdir().unwrap();
let admindir = dir.path();
// Cross build: armhf packages built ON an amd64 machine, so
// DEB_HOST_ARCH=armhf but DEB_BUILD_ARCH=amd64.
let opts = CheckOpts {
host_arch: "armhf".to_string(),
build_arch: "amd64".to_string(),
admindir: admindir.to_path_buf(),
..Default::default()
};
let control_for = |bd: &str| {
ControlInfo::parse_content(&format!(
"Source: t\nMaintainer: a <a@b.c>\nBuild-Depends: {bd}\n\nPackage: t\nArchitecture: any\nDescription: x\n y\n"
))
.unwrap()
};
// foo is only installed for the build architecture, like a native
// toolchain package pulled in on the build machine.
std::fs::write(
admindir.join("status"),
"\
Package: foo
Status: install ok installed
Version: 1.0
Architecture: amd64
",
)
.unwrap();
// `:native` resolves against the BUILD architecture: satisfied even
// though the host architecture is armhf.
let report = check_build_depends(&control_for("foo:native"), &opts).unwrap();
assert!(report.is_ok());
// Bracketed architecture restrictions keep evaluating against the
// HOST architecture: the amd64 instance cannot satisfy `foo [armhf]`.
// (The applied restriction reduces away, per dpkg's reduce_arch.)
let report = check_build_depends(&control_for("foo [armhf]"), &opts).unwrap();
assert_eq!(report.message(), "unmet build dependencies: foo");
// Now foo is only installed for the host architecture.
std::fs::write(
admindir.join("status"),
"\
Package: foo
Status: install ok installed
Version: 1.0
Architecture: armhf
",
)
.unwrap();
// `:native` no longer matches: no amd64 instance is installed.
let report = check_build_depends(&control_for("foo:native"), &opts).unwrap();
assert_eq!(report.message(), "unmet build dependencies: foo:native");
// The host-arch restriction matches the armhf instance.
let report = check_build_depends(&control_for("foo [armhf]"), &opts).unwrap();
assert!(report.is_ok());
}
}
+6 -4
View File
@@ -20,10 +20,12 @@ pub mod files;
pub mod version;
pub use changelog::{
ChangelogEntry, parse_changelog_entry, parse_changelog_entry_from_str,
parse_previous_version_from_str,
ChangelogEntry, parse_changelog_entries, parse_changelog_entries_from_str,
parse_changelog_entry, parse_changelog_entry_from_str,
};
pub use checksums::{ChecksumKind, Entry as ChecksumEntry, FileChecksums};
pub use control::{
ControlInfo, Paragraph, parse_paragraphs, strip_clearsigned_armour, write_paragraph,
};
pub use checksums::{Entry as ChecksumEntry, FileChecksums};
pub use control::{ControlInfo, Paragraph, parse_paragraphs, write_paragraph};
pub use files::{FilesEntry, FilesList};
pub use version::DebianVersion;
+27
View File
@@ -48,6 +48,12 @@ impl DebianVersion {
}
}
if let Some(rev) = &debian_revision {
if rev.is_empty() {
// dpkg rejects a trailing hyphen: "bad syntax: revision
// number is empty". Native versions (no `-` at all) are
// handled above and stay valid.
return Err(format!("empty debian revision in '{}'", raw));
}
for c in rev.chars() {
if !(c.is_ascii_alphanumeric() || matches!(c, '.' | '+' | '~') || !c.is_ascii()) {
return Err(format!(
@@ -329,6 +335,27 @@ mod tests {
assert!(DebianVersion::parse("1.0~rc1-2").is_ok());
}
/// dpkg rejects a trailing `-` ("bad syntax: revision number is
/// empty") but accepts `1.0--1`, where the revision is the text after
/// the *last* hyphen (upstream `1.0-` + revision `1`).
#[test]
fn version_empty_revision() {
let err = DebianVersion::parse("1.0-").unwrap_err();
assert!(err.contains("empty"), "unexpected message: {err}");
assert!(DebianVersion::parse("1.0-").is_err());
// Epoch variants take the same path.
assert!(DebianVersion::parse("3:1.0-").is_err());
assert!(DebianVersion::parse("1.0-1").is_ok());
// Native versions (no revision at all) are still fine.
assert!(DebianVersion::parse("1.0").is_ok());
assert!(DebianVersion::parse("3:1.0").is_ok());
let v = DebianVersion::parse("1.0--1").unwrap();
assert_eq!(v.upstream, "1.0-");
assert_eq!(v.debian_revision.as_deref(), Some("1"));
}
fn cmp_sign(a: &DebianVersion, b: &DebianVersion) -> i32 {
match a.cmp(b) {
std::cmp::Ordering::Less => -1,
+761 -48
View File
@@ -1,8 +1,11 @@
use crate::data::embed_data;
use chrono::NaiveDate;
use lazy_static::lazy_static;
use serde::Deserialize;
use std::collections::HashMap;
use std::error::Error;
use std::path::Path;
use std::time::Duration;
#[derive(Debug, Clone)]
/// Information about a specific distribution series
@@ -13,8 +16,8 @@ pub struct SeriesInformation {
pub codename: String,
/// Series version as numbers
pub version: Option<String>,
/// Series creation date
pub created: NaiveDate,
/// Series creation date (absent if missing or invalid in the CSV data)
pub created: Option<NaiveDate>,
/// Series release date
pub release: Option<NaiveDate>,
/// Series end-of-life date
@@ -27,22 +30,162 @@ struct SeriesInfo {
network: String,
}
/// Architectures an archive mirror serves: an explicit list, or the `all`
/// sentinel meaning one mirror serves every architecture (Debian's mirror
/// setup — an exhaustive list would rot each time an arch is added)
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum MirrorArchs {
/// The `all` sentinel
All(String),
/// An explicit list of dpkg architecture names
List(Vec<String>),
}
impl MirrorArchs {
/// Whether the mirror serves `arch`. A scalar other than the `all`
/// sentinel serves nothing (a test locks that reading of the data).
fn serves(&self, arch: &str) -> bool {
match self {
MirrorArchs::All(sentinel) => sentinel == "all",
MirrorArchs::List(archs) => archs.iter().any(|a| a == arch),
}
}
}
/// One archive mirror of a distribution: a URL serving a set of
/// architectures, plus the sibling host serving its `-security` pocket
/// for the same architectures (ports mirrors serve their own security)
#[derive(Debug, Deserialize)]
pub struct Mirror {
/// Base URL of the mirror (the primary mirror's URL doubles as the
/// dist's base URL, see [`get_base_url`])
pub url: String,
/// Sibling host serving the `-security` pocket for these
/// architectures; `None` when the mirror serves its own security
#[serde(default)]
security_url: Option<String>,
/// Architectures the mirror serves (see [`MirrorArchs`])
archs: MirrorArchs,
}
impl Mirror {
/// Whether the mirror serves `arch`
pub fn serves(&self, arch: &str) -> bool {
self.archs.serves(arch)
}
}
#[derive(Debug, Deserialize)]
struct DistData {
base_url: String,
mirrors: HashMap<String, Mirror>,
archive_keyring: String,
pockets: Vec<String>,
#[serde(default)]
sections: Vec<String>,
components: Vec<String>,
cross_pockets: Vec<String>,
#[serde(default)]
build_profiles: Vec<String>,
/// Changelog suite names aliasing a distro-info series codename
/// ('unstable' for Debian's 'sid'): the two names identify the same
/// series ([`series_suite_alias`], [`resolve_suite_alias`])
#[serde(default)]
suite_aliases: HashMap<String, String>,
series: SeriesInfo,
}
#[derive(Debug, Deserialize)]
struct Data {
dist: std::collections::HashMap<String, DistData>,
dist: HashMap<String, DistData>,
}
embed_data! {
static ref DATA: Data = "../data/distro_info.yml"
}
const DATA_YAML: &str = include_str!("../distro_info.yml");
lazy_static! {
static ref DATA: Data = serde_yaml::from_str(DATA_YAML).unwrap();
// Shared HTTP client used for all outgoing plain requests: timeouts keep
// a hanging remote (connect or transfer) from stalling pkh indefinitely.
// The short pool idle timeout and TCP keepalive avoid reusing keep-alive
// connections that the remote closed in the meantime, which surfaces as
// spurious 'error sending request' failures on busy mirrors/CDNs.
static ref HTTP_CLIENT: reqwest::Client = reqwest::Client::builder()
.connect_timeout(Duration::from_secs(10))
.timeout(Duration::from_secs(30))
.pool_idle_timeout(Duration::from_secs(10))
.tcp_keepalive(Duration::from_secs(30))
.build()
.expect("building the shared HTTP client with static options cannot fail");
}
/// Shared HTTP client with a connect timeout (10s) and a total request
/// timeout (30s), to be used for all outgoing plain HTTP(S) requests
pub(crate) fn http_client() -> &'static reqwest::Client {
&HTTP_CLIENT
}
/// GET `url` with bounded retries on transient transport errors (a pooled
/// keep-alive connection closed by the remote, a momentary network hiccup,
/// ...): these always succeed again on a fresh connection, and mirrors are
/// busy enough that unguarded single attempts make bulk operations flaky.
///
/// The response status is not inspected: 404s and the like are meaningful
/// answers, not transport failures.
pub(crate) async fn http_get_retried(url: &str) -> reqwest::Result<reqwest::Response> {
http_get_retried_with_timeout(url, None).await
}
/// [`http_get_retried`] with a per-request timeout override, for large
/// streaming downloads that exceed the shared client's total timeout
pub(crate) async fn http_get_retried_with_timeout(
url: &str,
timeout: Option<Duration>,
) -> reqwest::Result<reqwest::Response> {
const ATTEMPTS: u32 = 3;
let mut last_error: Option<reqwest::Error> = None;
for attempt in 0..ATTEMPTS {
let mut request = http_client().get(url);
if let Some(timeout) = timeout {
request = request.timeout(timeout);
}
match request.send().await {
Ok(response) => return Ok(response),
Err(e) => {
if attempt + 1 < ATTEMPTS {
log::debug!(
"GET '{url}' failed (attempt {}/{}, retrying): {}",
attempt + 1,
ATTEMPTS,
e
);
tokio::time::sleep(Duration::from_millis(300 * (u64::from(attempt) + 1))).await;
}
last_error = Some(e);
}
}
}
Err(last_error.expect("at least one attempt was made"))
}
/// Parse an optional '%Y-%m-%d' date from a CSV cell, warning instead of
/// panicking on invalid remote data
fn parse_optional_date(value: Option<&str>, series: &str, field: &str) -> Option<NaiveDate> {
value.and_then(
|date_str| match NaiveDate::parse_from_str(date_str, "%Y-%m-%d") {
Ok(date) => Some(date),
Err(e) => {
log::warn!(
"Invalid '{}' date '{}' for series '{}': {}. Ignoring the date.",
field,
date_str,
series,
e
);
None
}
},
)
}
fn parse_series_csv(content: &str) -> Result<Vec<SeriesInformation>, Box<dyn Error>> {
@@ -79,24 +222,39 @@ fn parse_series_csv(content: &str) -> Result<Vec<SeriesInformation>, Box<dyn Err
let mut series_info_list = Vec::new();
for result in rdr.records() {
let record = result?;
let series = record.get(series_idx).unwrap().to_string();
let codename = record.get(codename_idx).unwrap().to_string();
let record = match result {
Ok(record) => record,
Err(e) => {
log::warn!("Skipping malformed series CSV row: {}", e);
continue;
}
};
// Rows missing essential identification fields are skipped: they
// cannot be used nor reported meaningfully. Dates, on the other
// hand, are all optional in the model, so a bad date keeps the row.
let Some(series) = record.get(series_idx).filter(|s| !s.is_empty()) else {
log::warn!(
"Skipping series CSV row without a 'series' value: {:?}",
record
);
continue;
};
let Some(codename) = record.get(codename_idx).filter(|s| !s.is_empty()) else {
log::warn!(
"Skipping series CSV row for series '{}' without a 'codename' value",
series
);
continue;
};
let version = record.get(version_idx).map(|s| s.to_string());
let created = record
.get(created_idx)
.map(|date_str| NaiveDate::parse_from_str(date_str, "%Y-%m-%d").unwrap())
.unwrap();
let release = record
.get(release_idx)
.map(|date_str| NaiveDate::parse_from_str(date_str, "%Y-%m-%d").unwrap());
let eol = record
.get(eol_idx)
.map(|date_str| NaiveDate::parse_from_str(date_str, "%Y-%m-%d").unwrap());
let created = parse_optional_date(record.get(created_idx), series, "created");
let release = parse_optional_date(record.get(release_idx), series, "release");
let eol = parse_optional_date(record.get(eol_idx), series, "eol");
series_info_list.push(SeriesInformation {
series,
codename,
series: series.to_string(),
codename: codename.to_string(),
version,
created,
release,
@@ -110,9 +268,40 @@ fn parse_series_csv(content: &str) -> Result<Vec<SeriesInformation>, Box<dyn Err
Ok(series_info_list)
}
/// List the distributions known to pkh (e.g. "debian", "ubuntu")
/// List the distributions known to pkh (e.g. "debian", "ubuntu"), sorted so
/// that menus and error messages derived from it are deterministic
pub fn supported_dists() -> Vec<String> {
DATA.dist.keys().cloned().collect()
let mut dists: Vec<String> = DATA.dist.keys().cloned().collect();
dists.sort();
dists
}
/// Name of a dist's primary mirror entry: its URL doubles as the dist's
/// base URL ([`get_base_url`]) and is the first candidate of
/// [`mirror_for_arch`]
const PRIMARY_MIRROR: &str = "primary";
/// The data of a known distribution: the shared "unknown distribution"
/// error of the per-dist accessors
fn dist_data(dist: &str) -> Result<&'static DistData, Box<dyn Error>> {
DATA.dist.get(dist).ok_or_else(|| {
format!(
"Unknown distribution '{}'. Supported distributions are: {}.",
dist,
supported_dists().join(", ")
)
.into()
})
}
/// Special changelog distribution marking an entry that has not been
/// released to any archive series yet
pub const UNRELEASED: &str = "UNRELEASED";
/// Whether `series` is the special [`UNRELEASED`] distribution rather than
/// a real archive series
pub fn is_unreleased(series: &str) -> bool {
series == UNRELEASED
}
/// Get time-ordered list of series information for a distribution, development series first
@@ -126,7 +315,7 @@ pub async fn get_ordered_series(dist: &str) -> Result<Vec<SeriesInformation>, Bo
})?;
let series_info = &dist_data.series;
let content = if Path::new(series_info.local.as_str()).exists() {
std::fs::read_to_string(format!("/usr/share/distro-info/{dist}.csv")).map_err(|e| {
std::fs::read_to_string(series_info.local.as_str()).map_err(|e| {
format!(
"Failed to read distribution series data for '{dist}' \
from '{}': {}. The 'distro-info' package provides these CSV files.",
@@ -134,7 +323,9 @@ pub async fn get_ordered_series(dist: &str) -> Result<Vec<SeriesInformation>, Bo
)
})?
} else {
reqwest::get(series_info.network.as_str())
http_client()
.get(series_info.network.as_str())
.send()
.await?
.text()
.await?
@@ -150,6 +341,30 @@ pub async fn get_ordered_series_name(dist: &str) -> Result<Vec<String>, Box<dyn
Ok(series.iter().map(|info| info.series.clone()).collect())
}
/// The series to actually target when the changelog says [`UNRELEASED`]:
/// the development series of `dist`, i.e. the first entry of
/// [`get_ordered_series_name`] (which is documented "development series
/// first"). UNRELEASED work conventionally targets the next release, not
/// the last stable one. `dist` is matched case-insensitively, so vendor
/// names with original casing (dpkg's `Vendor:` field is e.g. "Ubuntu")
/// are accepted as-is. Any other `series` is returned unchanged. Errors
/// when `dist` is unknown or has no series list.
pub async fn effective_series(series: &str, dist: &str) -> Result<String, Box<dyn Error>> {
if !is_unreleased(series) {
return Ok(series.to_string());
}
// The series data keys are lowercase, unlike the vendor names that
// callers typically resolve from dpkg
let dist = dist.to_lowercase();
get_ordered_series_name(&dist)
.await?
.into_iter()
.next()
.ok_or_else(|| format!("Distribution '{dist}' has no series to target").into())
}
/// Get the latest released series for a dist (excluding future releases and special cases like sid)
pub async fn get_latest_released_series(dist: &str) -> Result<String, Box<dyn Error>> {
let latest = get_n_latest_released_series(dist, 1).await?;
@@ -203,25 +418,80 @@ pub async fn get_dist_from_series(series: &str) -> Result<String, Box<dyn Error>
Err(format!("Unknown series: {}", series).into())
}
/// Get the package pockets available for a given distribution
///
/// Example: get_dist_pockets(ubuntu) => ["proposed", "updates", ""]
pub fn get_dist_pockets(dist: &str) -> Result<Vec<String>, Box<dyn Error>> {
let dist_data = DATA.dist.get(dist).ok_or_else(|| {
format!(
"Unknown distribution '{}'. Supported distributions are: {}.",
dist,
supported_dists().join(", ")
)
})?;
let mut pockets = dist_data.pockets.clone();
/// The changelog suite name that aliases the series codename of `dist`
/// (Debian's 'unstable' for 'sid'): the two names identify the same
/// series. `None` when the series carries no suite alias.
pub fn series_suite_alias(dist: &str, series: &str) -> Option<String> {
dist_data(dist)
.ok()?
.suite_aliases
.iter()
.find(|(_suite, codename)| codename.as_str() == series)
.map(|(suite, _)| suite.clone())
}
// Explicitely add 'main' pocket, which is just the empty string
pockets.push("".to_string());
/// Identify a changelog suite name with the distro-info series codename
/// it aliases (Debian's 'unstable' is 'sid'), and the dist that codename
/// belongs to. `None` when `suite` is not a known alias of any dist.
pub fn resolve_suite_alias(suite: &str) -> Option<(String, String)> {
for (dist, data) in DATA.dist.iter() {
if let Some(codename) = data.suite_aliases.get(suite) {
return Some((dist.clone(), codename.clone()));
}
}
None
}
/// Get the package pockets available for a given distribution, in search order
///
/// The main archive ('') comes first so that a search without an explicit
/// pocket prefers the released archive over its pockets; development pockets
/// (e.g. '-proposed') come last.
///
/// Example: get_dist_pockets(ubuntu) => ["", "updates", "security", "proposed"]
pub fn get_dist_pockets(dist: &str) -> Result<Vec<String>, Box<dyn Error>> {
let mut pockets = dist_data(dist)?.pockets.clone();
// Explicitely add 'main' pocket, which is just the empty string, first
pockets.insert(0, "".to_string());
Ok(pockets)
}
/// Get the archive components of a distribution (ubuntu's main,
/// restricted, universe, multiverse; Debian's main, contrib, non-free,
/// non-free-firmware): the default set a build environment enables on its
/// official sources. Live archive operations keep resolving components
/// from Release files ([`get_components`]); this is the offline default.
pub fn get_dist_components(dist: &str) -> Result<Vec<String>, Box<dyn Error>> {
Ok(dist_data(dist)?.components.clone())
}
/// Get the pockets a cross-build environment enables for a series (the
/// `<series>-<pocket>` suite list is built from these): updates,
/// backports and security. Deliberately separate from
/// [`get_dist_pockets`], which is the *search order* of pull — folding
/// backports into it would change pull behavior.
pub fn get_cross_pockets(dist: &str) -> Result<Vec<String>, Box<dyn Error>> {
Ok(dist_data(dist)?.cross_pockets.clone())
}
/// Get the default build profiles of a distribution's vendor (Ubuntu
/// activates `derivative.ubuntu noudeb`, Debian none), mirroring what
/// `Dpkg::BuildProfiles` resolves when `DEB_BUILD_PROFILES` is unset.
/// Vendors are matched case-insensitively by the caller (dpkg's `Vendor:`
/// field keeps its original casing).
pub fn get_build_profiles(dist: &str) -> Result<Vec<String>, Box<dyn Error>> {
Ok(dist_data(dist)?.build_profiles.clone())
}
/// Get the valid `Section` values of a distribution's packages, as accepted
/// by its archives (a `section/subsection` in debian/control validates on
/// the part before the '/')
pub fn get_sections(dist: &str) -> Result<Vec<String>, Box<dyn Error>> {
Ok(dist_data(dist)?.sections.clone())
}
/// Get the sources URL for a distribution, series, pocket, and component
pub fn get_sources_url(base_url: &str, series: &str, pocket: &str, component: &str) -> String {
let pocket_full = if pocket.is_empty() {
@@ -232,23 +502,105 @@ pub fn get_sources_url(base_url: &str, series: &str, pocket: &str, component: &s
format!("{base_url}/dists/{series}{pocket_full}/{component}/source/Sources.gz")
}
/// Get the archive base URL for a distribution
/// Get the archive base URL for a distribution: the URL of its primary
/// mirror (the former `base_url` key folded into `mirrors.primary.url`
/// when the mirrors were modeled — the signature is kept so the pull
/// paths do not churn)
///
/// Example: ubuntu => http://archive.ubuntu.com/ubuntu
/// Example: ubuntu => https://archive.ubuntu.com/ubuntu
pub fn get_base_url(dist: &str) -> Result<String, Box<dyn Error>> {
DATA.dist
.get(dist)
.map(|d| d.base_url.clone())
let mirror = dist_data(dist)?
.mirrors
.get(PRIMARY_MIRROR)
.ok_or_else(|| {
format!(
"Unknown distribution '{}'. Supported distributions are: {}.",
dist,
"Distribution '{dist}' has no '{PRIMARY_MIRROR}' mirror in the built-in \
configuration. This is a bug; supported distributions are: {}.",
supported_dists().join(", ")
)
})?;
Ok(mirror.url.clone())
}
/// The mirror of `dist` serving `arch`: the primary mirror first, then
/// the other mirrors by name, so the answer is deterministic. Debian's
/// `all` sentinel makes its primary mirror serve every architecture.
/// Errors when the dist is unknown or no mirror serves the architecture
/// (an architecture the built-in data does not know about).
///
/// Example: mirror_for_arch(ubuntu, riscv64) => the ports mirror
pub fn mirror_for_arch(dist: &str, arch: &str) -> Result<&'static Mirror, Box<dyn Error>> {
let data = dist_data(dist)?;
if let Some(primary) = data.mirrors.get(PRIMARY_MIRROR)
&& primary.serves(arch)
{
return Ok(primary);
}
let mut others: Vec<&String> = data
.mirrors
.keys()
.filter(|name| name.as_str() != PRIMARY_MIRROR)
.collect();
others.sort();
others
.into_iter()
.filter_map(|name| data.mirrors.get(name))
.find(|mirror| mirror.serves(arch))
.ok_or_else(|| {
format!(
"No mirror of '{dist}' serves the '{arch}' architecture. Supported \
distributions are: {}.",
supported_dists().join(", ")
)
.into()
})
}
/// Host part of an apt-source URL: everything after the `://` scheme up
/// to the first `/` (a `:port` suffix stripped). URLs without a scheme
/// yield their leading segment.
fn url_host(url: &str) -> &str {
let after_scheme = url.split_once("://").map_or(url, |(_, rest)| rest);
let authority = after_scheme.split('/').next().unwrap_or(after_scheme);
authority
.split_once(':')
.map_or(authority, |(host, _)| host)
}
/// Whether the host of `uri` is the host of `mirror_url` or a subdomain
/// of it: the old substring checks (`uri.contains("archive.ubuntu.com")`)
/// intentionally matched the country mirrors fronting each archive host
/// (`fr.archive.ubuntu.com`), and an exact host comparison would have
/// dropped them. The leading dot of the suffix keeps look-alike hosts
/// (`notarchive.ubuntu.com`) out.
fn uri_matches_url_host(uri: &str, mirror_url: &str) -> bool {
let host = url_host(mirror_url);
let uri_host = url_host(uri);
uri_host == host || uri_host.ends_with(&format!(".{host}"))
}
/// Whether `uri` points at `mirror` or its security sibling: the URI's
/// host is the mirror's (or the sibling's) host or a subdomain of it (see
/// [`uri_matches_url_host`])
pub fn is_mirror_source(mirror: &Mirror, uri: &str) -> bool {
uri_matches_url_host(uri, &mirror.url)
|| mirror
.security_url
.as_deref()
.is_some_and(|security| uri_matches_url_host(uri, security))
}
/// Whether `uri` points at an official source of `dist` — one of its
/// archive mirrors or their security siblings — as opposed to a PPA or
/// another third-party repository. Unknown distributions match nothing.
pub fn is_official_source(dist: &str, uri: &str) -> bool {
DATA.dist.get(dist).is_some_and(|data| {
data.mirrors
.values()
.any(|mirror| is_mirror_source(mirror, uri))
})
}
/// Obtain the URLs for the archive keyrings of a distribution series
///
/// For 'sid' and 'experimental', returns keyrings from the 3 latest releases
@@ -320,7 +672,7 @@ pub async fn get_components(
let url = get_release_url(base_url, series, pocket);
log::debug!("Fetching Release file from: {}", url);
let content = reqwest::get(&url).await?.text().await?;
let content = http_client().get(&url).send().await?.text().await?;
for line in content.lines() {
if line.starts_with("Components:")
@@ -355,7 +707,9 @@ pub async fn get_debian_series_number(series: &str) -> Result<Option<String>, Bo
)
})?
} else {
reqwest::get(series_info.network.as_str())
http_client()
.get(series_info.network.as_str())
.send()
.await?
.text()
.await?
@@ -387,10 +741,315 @@ pub async fn get_debian_series_number(series: &str) -> Result<Option<String>, Bo
Ok(None)
}
/// The release number of a distribution series, paired with the dist it
/// belongs to: the version column of the series data, stripped to its
/// leading token ("12" for Debian bookworm, "26.04" out of Ubuntu
/// resolute's "26.04 LTS"). `None` when the series carries no version at
/// all (Debian's rolling sid/experimental have an empty column;
/// pseudo-versions like "unstable" pass through, callers validate per
/// vendor). Errors when no known distribution carries the series.
pub async fn get_series_release_number(
series: &str,
) -> Result<Option<(String, String)>, Box<dyn Error>> {
let dist = get_dist_from_series(series).await?;
for info in get_ordered_series(&dist).await? {
if info.series == series {
let number = info
.version
.as_deref()
.and_then(|version| version.split_whitespace().next())
.map(str::to_string);
return Ok(number.map(|number| (dist, number)));
}
}
Ok(None)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_get_sections() {
// Both distributions bundle the policy section list
for dist in ["debian", "ubuntu"] {
let sections = get_sections(dist).unwrap();
assert!(sections.contains(&"utils".to_string()));
assert!(sections.contains(&"devel".to_string()));
// 'unknown' is exactly what archives reject
assert!(!sections.contains(&"unknown".to_string()));
}
assert!(get_sections("not-a-distro").is_err());
}
/// The primary mirror's URL is the former `base_url`, byte for byte:
/// the pull paths build their archive URLs from it.
#[test]
fn test_primary_mirror_url_is_the_base_url() {
assert_eq!(
get_base_url("ubuntu").unwrap(),
"https://archive.ubuntu.com/ubuntu"
);
assert_eq!(
get_base_url("debian").unwrap(),
"https://deb.debian.org/debian"
);
assert!(get_base_url("not-a-distro").is_err());
}
/// Mirror-per-architecture resolution: the local architectures come
/// from Ubuntu's primary mirror, the others from ports; Debian's `all`
/// sentinel makes its one mirror serve everything, including
/// architectures the data never lists.
#[test]
fn test_mirror_for_arch() {
assert_eq!(
mirror_for_arch("ubuntu", "amd64").unwrap().url,
"https://archive.ubuntu.com/ubuntu"
);
assert_eq!(
mirror_for_arch("ubuntu", "riscv64").unwrap().url,
"http://ports.ubuntu.com/ubuntu-ports"
);
for arch in ["amd64", "riscv64", "brand-new"] {
assert_eq!(
mirror_for_arch("debian", arch).unwrap().url,
"https://deb.debian.org/debian",
"the `all` sentinel serves every architecture, including {arch}"
);
}
// An architecture no Ubuntu mirror serves, and an unknown dist.
assert!(mirror_for_arch("ubuntu", "mips64el").is_err());
assert!(mirror_for_arch("not-a-distro", "amd64").is_err());
}
/// The `archs` forms and their reading: the `all` sentinel serves
/// everything, an explicit list serves exactly its members, and any
/// other scalar serves nothing (a data bug a validation would have to
/// catch, hence the documented reading).
#[test]
fn test_mirror_archs_forms() {
let all: MirrorArchs = serde_yaml::from_str("all").unwrap();
assert!(all.serves("anything"));
let list: MirrorArchs = serde_yaml::from_str("[amd64, i386]").unwrap();
assert!(list.serves("amd64"));
assert!(!list.serves("arm64"));
let typo: MirrorArchs = serde_yaml::from_str("every").unwrap();
assert!(!typo.serves("amd64"));
}
/// Official-source matching is host-based but keeps matching the
/// country mirrors the old substring checks matched (`fr.archive.
/// ubuntu.com`): equality or a `.{host}` suffix, never a bare
/// substring — `notarchive.ubuntu.com` must not match.
#[test]
fn test_is_official_source_matches_country_mirrors_only() {
for uri in [
"https://archive.ubuntu.com/ubuntu",
"http://security.ubuntu.com/ubuntu",
"http://ports.ubuntu.com/ubuntu-ports",
// Country mirrors front the same archives.
"http://fr.archive.ubuntu.com/ubuntu",
"https://de.security.ubuntu.com/ubuntu",
] {
assert!(is_official_source("ubuntu", uri), "{uri}");
}
for uri in [
"https://deb.debian.org/debian",
"https://ppa.launchpadcontent.net/user/ppa/ubuntu",
"http://notarchive.ubuntu.com/ubuntu",
"http://archive.ubuntu.com.evil.example/ubuntu",
] {
assert!(!is_official_source("ubuntu", uri), "{uri}");
}
// Debian: its own mirror matches, Ubuntu's mirrors do not, and an
// unknown dist matches nothing.
assert!(is_official_source(
"debian",
"https://deb.debian.org/debian"
));
assert!(!is_official_source(
"debian",
"http://security.ubuntu.com/ubuntu"
));
assert!(!is_official_source(
"not-a-distro",
"https://deb.debian.org/debian"
));
}
/// The dist-level defaults the build paths read: components,
/// cross-build pockets (deliberately not the pull search order) and
/// vendor build profiles.
#[test]
fn test_dist_components_cross_pockets_and_build_profiles() {
assert_eq!(
get_dist_components("ubuntu").unwrap(),
vec!["main", "restricted", "universe", "multiverse"]
);
assert!(
get_dist_components("debian")
.unwrap()
.contains(&"non-free-firmware".to_string())
);
for dist in ["debian", "ubuntu"] {
assert_eq!(
get_cross_pockets(dist).unwrap(),
vec!["updates", "backports", "security"]
);
}
// Not the pull search order: no 'proposed', no empty main pocket.
let cross = get_cross_pockets("ubuntu").unwrap();
assert!(!cross.contains(&"proposed".to_string()));
assert!(!cross.contains(&"".to_string()));
assert_eq!(
get_build_profiles("ubuntu").unwrap(),
vec!["derivative.ubuntu", "noudeb"]
);
assert!(get_build_profiles("debian").unwrap().is_empty());
for getter in [
get_dist_components as fn(&str) -> Result<Vec<String>, Box<dyn Error>>,
get_cross_pockets as fn(&str) -> Result<Vec<String>, Box<dyn Error>>,
get_build_profiles as fn(&str) -> Result<Vec<String>, Box<dyn Error>>,
] {
assert!(getter("not-a-distro").is_err());
}
}
#[test]
fn test_parse_series_csv_malformed_rows() {
// A short row (missing 'codename') is skipped, a row with an invalid
// 'created' date is kept without a date, and invalid 'release'/'eol'
// dates become None: none of this may panic on remote data
let csv_data = "series,codename,version,created,release,eol\n\
noble,Noble N,24.04,2023-10-26,2024-04-25,2029-04-25\n\
lonely\n\
badbad,Bad B,1.0,not-a-date,2020-01-01,also-bad\n\
sid,sid,unstable,1999-01-01,,\n";
let series = parse_series_csv(csv_data).unwrap();
// Rows are returned most recent first (the parser reverses the list),
// with the malformed 'lonely' row skipped entirely
let names: Vec<&str> = series.iter().map(|s| s.series.as_str()).collect();
assert_eq!(names, vec!["sid", "badbad", "noble"]);
let noble = &series[2];
assert_eq!(noble.codename, "Noble N");
assert_eq!(noble.version.as_deref(), Some("24.04"));
assert_eq!(
noble.created,
Some(NaiveDate::from_ymd_opt(2023, 10, 26).unwrap())
);
assert_eq!(
noble.release,
Some(NaiveDate::from_ymd_opt(2024, 4, 25).unwrap())
);
assert_eq!(
noble.eol,
Some(NaiveDate::from_ymd_opt(2029, 4, 25).unwrap())
);
let badbad = &series[1];
assert_eq!(badbad.created, None);
assert_eq!(
badbad.release,
Some(NaiveDate::from_ymd_opt(2020, 1, 1).unwrap())
);
assert_eq!(badbad.eol, None);
}
#[test]
fn test_get_dist_pockets_order() {
// Without an explicit pocket, packages are searched in this order:
// main archive first, then updates, security, and proposed last
let pockets = get_dist_pockets("ubuntu").unwrap();
assert_eq!(
pockets,
vec![
"".to_string(),
"updates".to_string(),
"security".to_string(),
"proposed".to_string()
]
);
let pockets = get_dist_pockets("debian").unwrap();
assert_eq!(
pockets,
vec![
"".to_string(),
"updates".to_string(),
"security".to_string(),
"proposed-updates".to_string()
]
);
}
#[test]
fn test_is_unreleased() {
// Matching is exact: UNRELEASED is uppercase by Debian convention
assert!(is_unreleased("UNRELEASED"));
assert!(!is_unreleased("unreleased"));
assert!(!is_unreleased("noble"));
assert!(!is_unreleased(""));
}
#[tokio::test]
async fn test_effective_series_passthrough() {
// A real series is returned unchanged, and the dist is not even
// looked up (an unknown dist only matters for UNRELEASED)
assert_eq!(effective_series("noble", "ubuntu").await.unwrap(), "noble");
assert_eq!(effective_series("sid", "debian").await.unwrap(), "sid");
assert_eq!(
effective_series("noble", "unknown-distro").await.unwrap(),
"noble"
);
}
#[tokio::test]
async fn test_effective_series_unreleased() {
// UNRELEASED resolves to the development series of the dist, i.e.
// the first entry of the time-ordered list. On current distro-info
// data this is the next Ubuntu release, while Debian's list starts
// with 'experimental' (sid comes second), so assert against the
// data itself rather than a hardcoded name.
for dist in ["ubuntu", "debian"] {
let ordered = get_ordered_series_name(dist).await.unwrap();
let resolved = effective_series(UNRELEASED, dist).await.unwrap();
assert_eq!(resolved, ordered[0]);
assert_ne!(resolved, UNRELEASED);
}
}
#[tokio::test]
async fn test_effective_series_unreleased_dist_case_insensitive() {
// Distro data keys are lowercase but dpkg vendors keep original
// casing ("Ubuntu"): the UNRELEASED lookup must resolve both
let expected = effective_series(UNRELEASED, "ubuntu").await.unwrap();
assert_eq!(
effective_series(UNRELEASED, "Ubuntu").await.unwrap(),
expected
);
assert_eq!(
effective_series(UNRELEASED, "UBUNTU").await.unwrap(),
expected
);
}
#[tokio::test]
async fn test_effective_series_unknown_dist() {
// UNRELEASED on an unknown distribution cannot be resolved
assert!(
effective_series(UNRELEASED, "unknown-distro")
.await
.is_err()
);
}
#[tokio::test]
async fn test_get_debian_series() {
let series = get_ordered_series_name("debian").await.unwrap();
@@ -405,6 +1064,41 @@ mod tests {
assert!(series.contains(&"jammy".to_string()));
}
/// Suite aliases identify a changelog suite name with the series
/// codename of the same series: Debian's 'unstable' is 'sid'
#[test]
fn test_suite_aliases() {
assert_eq!(
resolve_suite_alias("unstable"),
Some(("debian".to_string(), "sid".to_string()))
);
// A series codename or unknown suite is not an alias
assert_eq!(resolve_suite_alias("sid"), None);
assert_eq!(resolve_suite_alias("noble"), None);
assert_eq!(
series_suite_alias("debian", "sid"),
Some("unstable".to_string())
);
assert_eq!(series_suite_alias("debian", "trixie"), None);
assert_eq!(series_suite_alias("ubuntu", "noble"), None);
}
/// Every suite alias must map to a real series of its dist, or the
/// selector would offer a phantom entry
#[tokio::test]
async fn test_suite_aliases_target_real_series() {
for (dist, data) in DATA.dist.iter() {
for (suite, codename) in &data.suite_aliases {
let series = get_ordered_series_name(dist).await.unwrap_or_default();
assert!(
series.contains(codename),
"suite alias '{suite}' of {dist} maps to '{codename}', \
which is not a known series"
);
}
}
}
#[tokio::test]
async fn test_get_dist_from_series() {
assert_eq!(get_dist_from_series("sid").await.unwrap(), "debian");
@@ -427,6 +1121,25 @@ mod tests {
assert!(unknown_number.is_none());
}
#[tokio::test]
async fn test_get_series_release_number() {
let (dist, bookworm) = get_series_release_number("bookworm")
.await
.unwrap()
.unwrap();
assert_eq!(dist, "debian");
assert_eq!(bookworm, "12");
// Ubuntu LTS rows carry a " LTS" decoration: only the leading
// YY.MM token is the release number
let (dist, noble) = get_series_release_number("noble").await.unwrap().unwrap();
assert_eq!(dist, "ubuntu");
assert_eq!(noble, "24.04");
// No known dist carries the series
assert!(get_series_release_number("not-a-series").await.is_err());
}
#[tokio::test]
async fn test_get_keyring_urls_sid() {
// Test that 'sid' returns keyrings from the 3 latest released versions
+310
View File
@@ -0,0 +1,310 @@
//! Passive interrupt state shared between the CLI and the library.
//!
//! Everything active about Ctrl+C lives in the CLI (`main.rs`): it installs
//! the SIGINT handler, wakes a watchdog thread, prints the interrupt notice
//! and exits with the conventional status 130. This module only holds the
//! state the library's own types need:
//!
//! - the interrupted flag ([`mark_interrupted`] / [`interrupted`]), read by
//! flows so they stand down while the watchdog tears everything down;
//! - the cleanup hook registry ([`register_cleanup_hook`]) for resources
//! that must not outlive the process (e.g. the ephemeral build chroot,
//! see [`crate::deb::ephemeral`]), drained and run by the CLI watchdog
//! right before exiting ([`run_cleanup_hooks`]);
//! - the reporter slot ([`set_reporter`]): the live build view registers
//! how to clear the terminal (and where the full log lives); the CLI
//! runs it as the first step of the shutdown.
//!
//! Nothing here installs signal handlers, prints or exits: a library
//! consumer embedding these types keeps its own signal disposition.
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::{Mutex, PoisonError};
/// How the live view reports an interrupt: it clears the terminal and
/// returns the log-file hint to print below the notice, if any
pub type Reporter = Box<dyn FnOnce() -> Option<String> + Send>;
/// A boxed, send-safe cleanup hook body
type CleanupFn = Box<dyn Fn() + Send>;
/// The reporter run before the cleanup hooks; taken out when it runs
static REPORTER: Mutex<Option<Reporter>> = Mutex::new(None);
/// Whether a Ctrl+C has been intercepted since the CLI installed the
/// handler
static INTERRUPTED: AtomicBool = AtomicBool::new(false);
/// Registry of cleanup hooks waiting to run at interrupt time
static CLEANUP_HOOKS: Mutex<Vec<CleanupHook>> = Mutex::new(Vec::new());
/// Source of the registry ids used to deregister a specific hook
static NEXT_CLEANUP_HOOK_ID: AtomicU64 = AtomicU64::new(1);
/// A pending cleanup hook together with its registry id
struct CleanupHook {
id: u64,
f: CleanupFn,
}
/// Record that a Ctrl+C has been intercepted; called by the CLI signal
/// handler
pub fn mark_interrupted() {
INTERRUPTED.store(true, Ordering::SeqCst);
}
/// Whether a Ctrl+C has been intercepted; flows use this to stay quiet and
/// to leave the cleanup to the CLI watchdog
pub fn interrupted() -> bool {
INTERRUPTED.load(Ordering::SeqCst)
}
/// Register how the live view reports an interrupt: the CLI watchdog runs
/// it as the first step of the shutdown, before the cleanup hooks. At most
/// one reporter runs per process: a later call replaces the one set before.
/// Without any reporter the watchdog only prints the plain notice.
pub fn set_reporter(report: Reporter) {
*REPORTER.lock().unwrap_or_else(PoisonError::into_inner) = Some(report);
}
/// Take the registered reporter out of the slot; `None` when no live view
/// registered one (`--verbose`, piped output)
pub fn take_reporter() -> Option<Reporter> {
REPORTER
.lock()
.unwrap_or_else(PoisonError::into_inner)
.take()
}
/// Register a hook to be run when the process is interrupted (after the
/// reporter), returning a guard whose drop deregisters the hook again.
///
/// Hooks must be self-contained — stored paths plus direct subprocesses —
/// and must never block indefinitely: they run in the watchdog while the
/// interrupted flow is still unwinding, and a second Ctrl+C during cleanup
/// is a no-op.
pub fn register_cleanup_hook(f: CleanupFn) -> CleanupHookGuard {
let id = NEXT_CLEANUP_HOOK_ID.fetch_add(1, Ordering::Relaxed);
CLEANUP_HOOKS
.lock()
.unwrap_or_else(PoisonError::into_inner)
.push(CleanupHook { id, f });
CleanupHookGuard(id)
}
/// RAII handle to a registered cleanup hook: dropping it (or an explicit
/// [`CleanupHookGuard::deregister`]) removes the hook from the registry so
/// the interrupt path can no longer run it
pub struct CleanupHookGuard(u64);
impl CleanupHookGuard {
/// Registry id of the hook (used to filter the registry in tests)
#[cfg(test)]
fn id(&self) -> u64 {
self.0
}
/// Remove the hook from the registry; returns whether it was still
/// pending
pub fn deregister(&mut self) -> bool {
deregister_cleanup_hook(self.0)
}
}
impl Drop for CleanupHookGuard {
fn drop(&mut self) {
deregister_cleanup_hook(self.0);
}
}
/// Remove a hook from the registry; returns whether it was still pending
fn deregister_cleanup_hook(id: u64) -> bool {
let mut hooks = CLEANUP_HOOKS.lock().unwrap_or_else(PoisonError::into_inner);
let len_before = hooks.len();
hooks.retain(|hook| hook.id != id);
hooks.len() != len_before
}
/// Drain and run every registered cleanup hook exactly once.
///
/// Called by the CLI watchdog right before the process exits. Draining uses
/// `try_lock` with a bounded retry instead of a blocking lock as a hard
/// upper bound on interrupt latency: the sequence must never hang waiting
/// for a lock, however unlikely a stalled holder is. Timing out therefore
/// skips cleanup (leaking) rather than hanging.
pub fn run_cleanup_hooks() {
run_drained_hooks(drain_cleanup_hooks());
}
/// Take every pending hook out of the registry, waiting at most ~1s for the
/// registry lock (see [`run_cleanup_hooks`] for why this must not block
/// forever)
fn drain_cleanup_hooks() -> Vec<CleanupHook> {
const RETRIES: usize = 200;
const RETRY_DELAY: std::time::Duration = std::time::Duration::from_millis(5);
for _ in 0..RETRIES {
if let Ok(mut hooks) = CLEANUP_HOOKS.try_lock() {
return std::mem::take(&mut *hooks);
}
std::thread::sleep(RETRY_DELAY);
}
log::error!("Timed out waiting for the cleanup hook registry; skipping interrupt cleanup");
Vec::new()
}
/// Run drained hooks one by one, isolating panics so that one failing hook
/// cannot skip the remaining ones
fn run_drained_hooks(hooks: Vec<CleanupHook>) {
for CleanupHook { id, f } in hooks {
// Hooks are arbitrary user code; assert unwind safety so they can be
// run inside a catching context
if let Err(panic) = std::panic::catch_unwind(std::panic::AssertUnwindSafe(f)) {
log::error!("Cleanup hook {id} panicked: {}", panic_message(&panic));
}
}
}
/// Best-effort message extraction from a panic payload
fn panic_message(panic: &(dyn std::any::Any + Send)) -> String {
if let Some(s) = panic.downcast_ref::<&str>() {
(*s).to_string()
} else if let Some(s) = panic.downcast_ref::<String>() {
s.clone()
} else {
"non-string panic payload".to_string()
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
use std::sync::Mutex as StdMutex;
use std::sync::atomic::AtomicUsize;
/// Serializes these tests: they drain the process-global registry, and
/// unrelated tests may hold registrations concurrently that must be
/// neither run nor lost. Poison-proof: a test failing while holding the
/// lock must not cascade into the others.
static TEST_LOCK: StdMutex<()> = StdMutex::new(());
fn test_lock() -> std::sync::MutexGuard<'static, ()> {
TEST_LOCK.lock().unwrap_or_else(PoisonError::into_inner)
}
/// Drain the registry and take out only the hooks with the given ids,
/// putting everything else back so unrelated registrations stay pending
fn take_hooks(ids: &[u64]) -> Vec<CleanupHook> {
let drained = drain_cleanup_hooks();
let mut mine = Vec::new();
let mut others = Vec::new();
for hook in drained {
if ids.contains(&hook.id) {
mine.push(hook);
} else {
others.push(hook);
}
}
CLEANUP_HOOKS
.lock()
.unwrap_or_else(PoisonError::into_inner)
.extend(others);
mine
}
/// Register a hook that counts its invocations
fn counting_hook() -> (CleanupHookGuard, Arc<AtomicUsize>) {
let counter = Arc::new(AtomicUsize::new(0));
let seen = counter.clone();
let guard = register_cleanup_hook(Box::new(move || {
seen.fetch_add(1, Ordering::SeqCst);
}));
(guard, counter)
}
/// Hooks run in registration order, and draining means each hook runs
/// exactly once even across repeated cleanup passes.
#[test]
fn hooks_run_once_in_registration_order() {
let _serial = test_lock();
let log = Arc::new(StdMutex::new(Vec::new()));
let mut guards = Vec::new();
let mut ids = Vec::new();
for name in ["hook-a", "hook-b", "hook-c"] {
let log = log.clone();
// The returned guard must stay alive: dropping it deregisters
let guard = register_cleanup_hook(Box::new(move || {
log.lock().unwrap().push(name);
}));
ids.push(guard.id());
guards.push(guard);
}
// Only our own hooks are extracted; they run in registration order
let mine = take_hooks(&ids);
assert_eq!(mine.len(), ids.len());
run_drained_hooks(mine);
assert_eq!(*log.lock().unwrap(), vec!["hook-a", "hook-b", "hook-c"]);
// Draining removed them: a second pass runs nothing again
assert!(take_hooks(&ids).is_empty());
assert_eq!(*log.lock().unwrap(), vec!["hook-a", "hook-b", "hook-c"]);
drop(guards);
}
/// A panicking hook is contained by the runner: it neither aborts the
/// process nor skips the hooks registered around it.
#[test]
fn panicking_hook_does_not_skip_the_others() {
let _serial = test_lock();
// The hook below panics on purpose: do not record it as a test
// failure in the end-of-run matrix
let _quiet = crate::test_support::suppress_failure_recording();
let (before, ran_before) = counting_hook();
let boom = register_cleanup_hook(Box::new(|| panic!("cleanup exploded")));
let (after, ran_after) = counting_hook();
let ids = [before.id(), boom.id(), after.id()];
run_drained_hooks(take_hooks(&ids));
assert_eq!(ran_before.load(Ordering::SeqCst), 1);
assert_eq!(ran_after.load(Ordering::SeqCst), 1);
}
/// Explicit deregistration removes the hook: it is no longer drained and
/// never runs; a second deregistration reports it as already gone.
#[test]
fn deregistered_hook_never_runs() {
let _serial = test_lock();
let (mut guard, ran) = counting_hook();
assert!(guard.deregister());
assert!(!guard.deregister());
assert!(take_hooks(&[guard.id()]).is_empty());
assert_eq!(ran.load(Ordering::SeqCst), 0);
}
/// Dropping the registration guard deregisters the hook implicitly.
#[test]
fn dropping_the_guard_deregisters_the_hook() {
let _serial = test_lock();
let id;
let ran;
{
let (guard, counter) = counting_hook();
id = guard.id();
ran = counter;
drop(guard);
}
assert!(take_hooks(&[id]).is_empty());
assert_eq!(ran.load(Ordering::SeqCst), 0);
}
}
+745
View File
@@ -0,0 +1,745 @@
//! Launchpad integration for `pkh put`: PPA upload targets, Launchpad
//! account (username) discovery and pre-upload checks against the Launchpad
//! API.
//!
//! Launchpad's SFTP upload server requires the SSH username to be a real
//! Launchpad account name — anonymous logins are rejected ("Launchpad user
//! 'anonymous' doesn't have a registered SSH key") — and authenticates it
//! with the SSH keys registered on that account
//! (<https://launchpad.net/~/+editsshkeys>). The username therefore has to
//! be discovered on the machine rather than hardcoded: first from the git
//! configuration ([`username`], the `lp.user` key), then through the generic
//! fallbacks (SSH configuration `User`, local user name — see
//! [`crate::put::ssh`]).
//!
//! The upload queue itself is a blind write: the SFTP server accepts any
//! file an authenticated user puts into their incoming area, and invalid
//! targets are only rejected later, during queue processing. The
//! [`ppa_info`] check makes sure the target actually exists before anything
//! is uploaded.
use std::error::Error;
use std::path::Path;
use serde::Deserialize;
use crate::data::embed_data;
use crate::put::target::UploadTarget;
/// Git configuration key holding the Launchpad account name
const LP_USER_KEY: &str = "lp.user";
/// Launchpad service endpoints, loaded from the bundled `launchpad.yml`
/// data file (same pattern as `distro_info.yml`): static endpoints that
/// change with Launchpad, not with the code, are data — several of them
/// were previously duplicated across three modules.
#[derive(Debug, Deserialize)]
struct LaunchpadData {
/// Base URL of the Launchpad REST API
api_base: String,
/// Host of the PPA SFTP upload server
ssh_host: String,
/// Port of the PPA SFTP upload server
ssh_port: u16,
/// Host of the PPA upload queue over anonymous FTP (the transport
/// `pkh put` degrades to when the SSH connection never comes up)
ftp_host: String,
/// Port of the anonymous FTP upload queue
ftp_port: u16,
/// Upload queue incoming directory template (`{owner}`/`{ppa}`)
incoming_template: String,
/// PPA package-content (apt repository) URL template
content_host_template: String,
/// Ubuntu source-package git web URL template (`{package}`)
git_web_template: String,
}
embed_data! {
static ref LAUNCHPAD_DATA: LaunchpadData = "../data/launchpad.yml"
}
/// The PPA upload queue over anonymous FTP (host, port): the transport
/// dput-ng's plain `ppa:` profile pushes over, and the one `pkh put`
/// degrades to when the SSH connection itself never comes up.
pub(crate) fn ppa_ftp_queue() -> (String, u16) {
(LAUNCHPAD_DATA.ftp_host.clone(), LAUNCHPAD_DATA.ftp_port)
}
/// Base URL of the Launchpad REST API
fn api_base() -> &'static str {
&LAUNCHPAD_DATA.api_base
}
/// Host serving PPA package content, derived from the content-host
/// template so the URL builders and the URL parsers of PPA addresses
/// cannot drift apart
pub(crate) fn ppa_content_host() -> &'static str {
let template = LAUNCHPAD_DATA.content_host_template.as_str();
let after_scheme = template
.split_once("://")
.map_or(template, |(_, rest)| rest);
after_scheme.split('/').next().unwrap_or(after_scheme)
}
/// Base URL of the apt repository serving a PPA's packages
/// (e.g. `https://ppa.launchpadcontent.net/user/ppa/ubuntu`)
pub(crate) fn ppa_content_url(owner: &str, ppa: &str) -> String {
LAUNCHPAD_DATA
.content_host_template
.replace("{owner}", owner)
.replace("{ppa}", ppa)
}
/// URL of the Launchpad git repository of an Ubuntu source package
/// (`git.launchpad.net/ubuntu/+source/<package>`), the preferred VCS of
/// Ubuntu packages
pub(crate) fn ubuntu_source_git_url(package: &str) -> String {
LAUNCHPAD_DATA
.git_web_template
.replace("{package}", package)
}
/// Page size (`ws.size`) asked from Launchpad collections. Launchpad
/// truncates collection answers at 75 entries by default and rejects
/// `ws.size` above 300 (both verified against the live API); 100 sits
/// comfortably under the cap while keeping multi-page walks rare.
const WS_PAGE_SIZE: u32 = 100;
/// Hard cap on the pages followed while walking a `getPublishedSources`
/// collection: 20 pages x 100 entries = 2000 currently published entries
/// for one source name. The query only counts `Published` entries of live
/// series/pockets, so real histories are a handful of entries; a walk
/// reaching the cap means the API is misbehaving (an endless next-link
/// chain), not that the history is genuinely huge.
const MAX_COLLECTION_PAGES: u32 = 20;
/// The Launchpad username configured in git: the repository-local
/// configuration wins over the global one, like git's own precedence.
/// `None` when no git repository is found or the key is unset.
pub fn username(cwd: &Path) -> Option<String> {
// A repository's config covers the local file; the global/system levels
// are consulted separately so the key is found in both setups
if let Ok(repo) = git2::Repository::discover(cwd)
&& let Ok(config) = repo.config()
&& let Some(value) = config_value(&config)
{
return Some(value);
}
if let Ok(config) = git2::Config::open_default() {
return config_value(&config);
}
None
}
/// Trimmed, non-empty `lp.user` value of a configuration, `None` when unset
fn config_value(config: &git2::Config) -> Option<String> {
config
.get_string(LP_USER_KEY)
.ok()
.map(|value| value.trim().to_string())
.filter(|value| !value.is_empty())
}
/// Split a `user/ppa_name` PPA argument, rejecting malformed ones
fn split_ppa(ppa: &str) -> Result<(String, String), String> {
let parts: Vec<&str> = ppa.split('/').collect();
if parts.len() != 2 || parts.iter().any(|p| p.is_empty()) {
return Err(format!(
"Invalid PPA format: '{ppa}'. Expected: user/ppa_name"
));
}
Ok((parts[0].to_string(), parts[1].to_string()))
}
/// URL of the Launchpad API resource of a Launchpad account
fn person_url(user: &str) -> String {
format!("{}/~{user}", api_base())
}
/// URL of the Launchpad API resource of a PPA (`~user/+archive/ubuntu/name`
/// covers the default `ppa` archive and named archives alike); shared by the
/// put-side pre-flight checks and the apt keyring's fingerprint lookup
pub(crate) fn archive_url(user: &str, ppa: &str) -> String {
format!("{}/~{user}/+archive/ubuntu/{ppa}", api_base())
}
/// Resolve a `user/ppa_name` PPA argument into its upload target (the
/// SFTP host and incoming template of `launchpad.yml`), like dput-ng's
/// `ppa:user/ppa` profile expansion.
pub fn ppa_target(ppa: &str) -> Result<UploadTarget, String> {
let (user, name) = split_ppa(ppa)?;
Ok(UploadTarget {
fqdn: LAUNCHPAD_DATA.ssh_host.clone(),
port: LAUNCHPAD_DATA.ssh_port,
login: None,
incoming: LAUNCHPAD_DATA
.incoming_template
.replace("{owner}", &user)
.replace("{ppa}", &name),
label: format!("ppa:{ppa}"),
})
}
/// The subset of the Launchpad Archive API resource relevant for uploads
#[derive(Debug, Deserialize)]
pub struct PpaInfo {
/// Display name of the archive (e.g. "Noctalia")
pub displayname: String,
/// The archive's self-description
pub description: Option<String>,
/// Disabled archives accept no uploads; absent/null on many archives
/// (treated as enabled)
pub enabled: Option<bool>,
}
/// Look up the PPA `user/name` (same format as `pkh put --ppa`) in the
/// Launchpad API, failing with a precise message when the account or the
/// archive does not exist, or the archive is disabled. This is the
/// pre-flight check the SFTP queue itself never does.
pub async fn ppa_info(ppa: &str) -> Result<PpaInfo, Box<dyn Error>> {
let (user, name) = split_ppa(ppa)?;
let client = crate::distro_info::http_client();
let response = client
.get(person_url(&user))
.send()
.await
.map_err(|e| format!("cannot reach the Launchpad API: {e}"))?;
if response.status() == reqwest::StatusCode::NOT_FOUND {
return Err(format!(
"Launchpad user '~{user}' does not exist: check the PPA argument '{ppa}'"
)
.into());
} else if !response.status().is_success() {
return Err(format!(
"Launchpad API returned {} for user '~{user}'",
response.status()
)
.into());
}
let response = client
.get(archive_url(&user, &name))
.send()
.await
.map_err(|e| format!("cannot reach the Launchpad API: {e}"))?;
match response.status() {
reqwest::StatusCode::OK => {
let info: PpaInfo = response
.json()
.await
.map_err(|e| format!("cannot parse the Launchpad API response for '{ppa}': {e}"))?;
if info.enabled == Some(false) {
return Err(
format!("PPA '{ppa}' is disabled: it exists but accepts no uploads").into(),
);
}
Ok(info)
}
reqwest::StatusCode::NOT_FOUND => {
Err(format!("PPA '{ppa}' does not exist: create it on launchpad.net first").into())
}
status => Err(format!("Launchpad API returned {status} for PPA '{ppa}'").into()),
}
}
/// Percent-encode a query-string value (RFC 3986): unreserved characters
/// pass through, everything else becomes `%XX`. Debian source package names
/// may contain `+` (`g++`), which must not reach the API unencoded — query
/// values follow form-urlencoded rules, where a literal `+` decodes to a
/// space.
fn percent_encode(value: &str) -> String {
let mut encoded = String::with_capacity(value.len());
for byte in value.bytes() {
if byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'.' | b'_' | b'~') {
encoded.push(byte as char);
} else {
encoded.push_str(&format!("%{byte:02X}"));
}
}
encoded
}
/// URL of the first page of the `getPublishedSources` API call listing the
/// currently `Published` source packages named `source_name` in the PPA
/// `user/name`: `exact_match` avoids Launchpad's default case-insensitive
/// substring matching, which would return unrelated sources (`data` matching
/// `datatables`). Further pages are reached through the answer's
/// `next_collection_link`, not by hand-building URLs.
fn published_sources_url(user: &str, ppa: &str, source_name: &str) -> String {
format!(
"{}?ws.op=getPublishedSources&source_name={}&exact_match=true&status=Published&ws.size={WS_PAGE_SIZE}",
archive_url(user, ppa),
percent_encode(source_name)
)
}
/// One page of a `getPublishedSources` answer: the subset of the source
/// package publishing history the superseded-upload check needs (the live
/// answer carries many more fields, ignored by serde)
#[derive(Debug, Deserialize)]
struct PublishedSource {
/// Version of the published source package
source_package_version: String,
}
/// One page of the `getPublishedSources` collection answer
#[derive(Debug, Deserialize)]
struct PublishedSources {
/// The currently published source packages matching the query, on this
/// page only
#[serde(default)]
entries: Vec<PublishedSource>,
/// URL of the next page, present only when the collection was
/// truncated (Launchpad answers carry it as a plain JSON string)
next_collection_link: Option<String>,
}
/// Parse one page of a `getPublishedSources` collection into the versions
/// it carries plus the link to the next page (`None` on the last one): the
/// pagination decision, factored out of the HTTP walk so it can be tested
/// without a server.
fn parse_collection_page(body: &str) -> Result<(Vec<String>, Option<String>), serde_json::Error> {
let sources: PublishedSources = serde_json::from_str(body)?;
Ok((
sources
.entries
.into_iter()
.map(|entry| entry.source_package_version)
.collect(),
sources.next_collection_link,
))
}
/// GET one page of a collection, mapping the API statuses to the same
/// errors as the other Launchpad calls (404 means the PPA does not exist).
/// Returns the response body for [`parse_collection_page`].
async fn fetch_collection_page(
client: &reqwest::Client,
url: &str,
ppa: &str,
) -> Result<String, Box<dyn Error>> {
let response = client
.get(url)
.send()
.await
.map_err(|e| format!("cannot reach the Launchpad API: {e}"))?;
match response.status() {
reqwest::StatusCode::OK => response
.text()
.await
.map_err(|e| format!("cannot read the Launchpad API response for '{ppa}': {e}").into()),
reqwest::StatusCode::NOT_FOUND => {
Err(format!("PPA '{ppa}' does not exist: create it on launchpad.net first").into())
}
status => Err(format!("Launchpad API returned {status} for PPA '{ppa}'").into()),
}
}
/// Walk a `getPublishedSources` collection page by page: fetch the first
/// page, then follow `next_collection_link` (the canonical Launchpad
/// pagination) until a page comes without one, accumulating the versions of
/// every page in order.
///
/// Exceeding [`MAX_COLLECTION_PAGES`] errors rather than returning the
/// partial list: the result feeds `put`'s superseded-upload check, where a
/// silently truncated list is exactly the bug pagination fixes — a
/// superseded upload wrongly allowed through, to be rejected (or to
/// silently supersede) in Launchpad's queue hours later. Every other
/// failure mode of this check (network, HTTP status, parsing) aborts the
/// upload too, and `put` fails before anything is written, so erring costs
/// only a clear message.
async fn walk_collection(
client: &reqwest::Client,
first_url: &str,
ppa: &str,
) -> Result<Vec<String>, Box<dyn Error>> {
let mut versions = Vec::new();
let mut url = first_url.to_string();
for _page in 1..=MAX_COLLECTION_PAGES {
let body = fetch_collection_page(client, &url, ppa).await?;
let (mut page_versions, next) = parse_collection_page(&body)
.map_err(|e| format!("cannot parse the Launchpad API response for '{ppa}': {e}"))?;
versions.append(&mut page_versions);
match next {
Some(next) => url = next,
None => return Ok(versions),
}
}
Err(format!(
"the Launchpad API keeps paginating the published sources of '{ppa}' \
after {MAX_COLLECTION_PAGES} pages: cannot run the superseded check \
on a partial list"
)
.into())
}
/// Every version of `source_name` currently `Published` in the PPA
/// `user/name` (same `user/ppa_name` format as `pkh put --ppa`), in API
/// order. Empty when the source was never published there — a 200 answer
/// with zero entries, the normal first-upload case. Launchpad truncates
/// collections per page, so the walk follows the API's `next_collection_link`
/// until the collection is exhausted: a single page would miss the highest
/// version of a source published in many series/pockets over time, and the
/// superseded check would wrongly pass.
pub async fn published_versions(
ppa: &str,
source_name: &str,
) -> Result<Vec<String>, Box<dyn Error>> {
let (user, name) = split_ppa(ppa)?;
walk_collection(
crate::distro_info::http_client(),
&published_sources_url(&user, &name, source_name),
ppa,
)
.await
}
/// PPA uploads only target Ubuntu series: fail before uploading when the
/// changes' distribution is not a known series (typo) or a non-Ubuntu one —
/// both are only rejected during queue processing otherwise
pub async fn check_ppa_series(distribution: &str) -> Result<(), Box<dyn Error>> {
match crate::distro_info::get_dist_from_series(distribution).await {
Ok(dist) if dist == "ubuntu" => Ok(()),
Ok(dist) => Err(format!(
"series '{distribution}' belongs to {dist}: PPA uploads target \
Ubuntu series only"
)
.into()),
Err(_) => Err(format!(
"'{distribution}' is not a known distribution series: check the \
debian/changelog entry, the upload would be rejected"
)
.into()),
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ppa_target_expands_user_and_name() {
let target = ppa_target("paultag/fluxbox").unwrap();
assert_eq!(target.fqdn, "ppa.launchpad.net");
assert_eq!(target.port, 22);
assert_eq!(target.incoming, "~paultag/fluxbox");
assert_eq!(target.label, "ppa:paultag/fluxbox");
// No static login: the username is discovered per machine
assert_eq!(target.login, None);
}
/// The anonymous FTP fallback queue resolves from the same data the
/// dput-ng `ppa:` profile uses.
#[test]
fn ppa_ftp_queue_resolves() {
assert_eq!(ppa_ftp_queue(), ("ppa.launchpad.net".to_string(), 21));
}
#[test]
fn ppa_target_rejects_missing_separator() {
assert!(ppa_target("just-a-name").is_err());
}
#[test]
fn ppa_target_rejects_extra_components() {
assert!(ppa_target("user/ppa/extra").is_err());
}
#[test]
fn ppa_target_rejects_empty_components() {
assert!(ppa_target("user/").is_err());
assert!(ppa_target("/ppa").is_err());
assert!(ppa_target("/").is_err());
}
/// The `lp.user` key is read from the git configuration of the
/// repository containing the working directory
#[test]
fn username_comes_from_repo_git_config() {
let dir = tempfile::tempdir().unwrap();
let repo = git2::Repository::init(dir.path()).unwrap();
repo.config()
.unwrap()
.set_str(LP_USER_KEY, "vhaudiquet")
.unwrap();
assert_eq!(username(dir.path()).as_deref(), Some("vhaudiquet"));
}
/// Values are trimmed, and an empty value counts as unset (it must not
/// shadow a real lookup failure with a useless username)
#[test]
fn username_ignores_blank_values() {
let dir = tempfile::tempdir().unwrap();
let repo = git2::Repository::init(dir.path()).unwrap();
repo.config().unwrap().set_str(LP_USER_KEY, " ").unwrap();
// Blank local value: the resolution keeps looking (and finds
// nothing here unless a global lp.user exists — the assertion
// accepts either "no value" or a real global value, never the
// blank one)
let found = username(dir.path());
assert_ne!(found.as_deref(), Some(" "));
let _ = found;
}
#[test]
fn api_urls_match_launchpad_resources() {
// Both URL shapes verified against the live API: 200 for an
// existing account/archive, 404 for a missing one
assert_eq!(
person_url("vhaudiquet"),
"https://api.launchpad.net/1.0/~vhaudiquet"
);
assert_eq!(
archive_url("vhaudiquet", "noctalia"),
"https://api.launchpad.net/1.0/~vhaudiquet/+archive/ubuntu/noctalia"
);
}
/// The data-driven endpoint accessors build the same addresses the
/// former hardcoded constants carried (each verified against the live
/// service), and the content host is derived from the same template
/// the content URLs are built from
#[test]
fn data_driven_endpoints_match_the_service() {
assert_eq!(
ppa_content_url("vhaudiquet", "noctalia"),
"https://ppa.launchpadcontent.net/vhaudiquet/noctalia/ubuntu"
);
assert_eq!(ppa_content_host(), "ppa.launchpadcontent.net");
assert_eq!(
ubuntu_source_git_url("hello"),
"https://git.launchpad.net/ubuntu/+source/hello"
);
}
/// The API answer carries many unrelated fields; deserialization must
/// pick the relevant ones and tolerate a null `enabled`
#[test]
fn ppa_info_parses_api_response() {
let json = r#"{
"self_link": "https://api.launchpad.net/1.0/~vhaudiquet/+archive/ubuntu/noctalia",
"web_link": "https://launchpad.net/~vhaudiquet/+archive/ubuntu/noctalia",
"displayname": "Noctalia",
"description": "Noctalia PPA with experimental builds",
"enabled": null,
"official_bug_tags": ["a11y", "appstream"]
}"#;
let info: PpaInfo = serde_json::from_str(json).unwrap();
assert_eq!(info.displayname, "Noctalia");
assert_eq!(
info.description.as_deref(),
Some("Noctalia PPA with experimental builds")
);
assert_eq!(info.enabled, None);
}
#[test]
fn percent_encode_keeps_unreserved_and_escapes_the_rest() {
// The characters of Debian source package names pass through
assert_eq!(percent_encode("noctalia"), "noctalia");
assert_eq!(percent_encode("libfoo-1.0"), "libfoo-1.0");
// `+` must be escaped: in query values it would decode to a space
assert_eq!(percent_encode("g++"), "g%2B%2B");
assert_eq!(percent_encode("a b/c?d&e"), "a%20b%2Fc%3Fd%26e");
}
/// The query matches the verified live `getPublishedSources` call, with
/// the source name percent-encoded and an explicit page size (the API
/// default of 75 entries would hide part of long publishing histories)
#[test]
fn published_sources_url_matches_launchpad_call() {
assert_eq!(
published_sources_url("vhaudiquet", "noctalia", "noctalia"),
"https://api.launchpad.net/1.0/~vhaudiquet/+archive/ubuntu/noctalia?ws.op=getPublishedSources&source_name=noctalia&exact_match=true&status=Published&ws.size=100"
);
assert_eq!(
published_sources_url("vhaudiquet", "noctalia", "g++"),
"https://api.launchpad.net/1.0/~vhaudiquet/+archive/ubuntu/noctalia?ws.op=getPublishedSources&source_name=g%2B%2B&exact_match=true&status=Published&ws.size=100"
);
}
/// The live answer carries many unrelated fields per entry; only
/// `source_package_version` is needed (shape verified against the API)
#[test]
fn published_sources_parses_api_response() {
let json = r#"{
"start": 0,
"total_size": 2,
"entries": [
{
"self_link": "https://api.launchpad.net/1.0/~vhaudiquet/+archive/ubuntu/noctalia/+sourcepub/18737497",
"resource_type_link": "https://api.launchpad.net/1.0/#source_package_publishing_history",
"display_name": "noctalia 5.1.0-1ubuntu2 in stonking",
"component_name": "main",
"section_name": "x11",
"status": "Published",
"pocket": "Release",
"date_published": "2026-09-16T19:36:46.116930+00:00",
"scheduled_deletion_date": null,
"source_package_name": "noctalia",
"source_package_version": "5.1.0-1ubuntu2",
"http_etag": "\"98f12b47\""
},
{"unknown_extra": {"nested": [1, 2]}, "source_package_version": "2:1.0-1"}
]
}"#;
let (versions, next) = parse_collection_page(json).unwrap();
assert_eq!(versions, vec!["5.1.0-1ubuntu2", "2:1.0-1"]);
// A page without a next link is the end of the collection
assert_eq!(next, None);
}
/// A 200 answer with zero entries is the normal "nothing published
/// there" case, and must deserialize to an empty list
#[test]
fn published_sources_parses_empty_collection() {
let (versions, next) =
parse_collection_page(r#"{"start": 0, "total_size": 0, "entries": []}"#).unwrap();
assert!(versions.is_empty());
assert_eq!(next, None);
}
/// A truncated page announces the next one through
/// `next_collection_link`, carried as a plain JSON string (shape
/// verified against the live API)
#[test]
fn parse_collection_page_reads_next_link() {
let json = r#"{
"start": 0,
"total_size": 150,
"entries": [{"source_package_version": "1.0-1"}],
"next_collection_link": "https://api.launchpad.net/1.0/~u/+archive/ubuntu/p?ws.op=getPublishedSources&ws.size=100&memo=100&ws.start=100"
}"#;
let (versions, next) = parse_collection_page(json).unwrap();
assert_eq!(versions, vec!["1.0-1"]);
assert_eq!(
next.as_deref(),
Some(
"https://api.launchpad.net/1.0/~u/+archive/ubuntu/p?ws.op=getPublishedSources&ws.size=100&memo=100&ws.start=100"
)
);
}
/// Serve canned byte responses on a local port, one per connection (the
/// last response repeats), and return the listener for URL building
///
/// The canned responses must use 'Connection: close' so the client opens
/// a fresh connection (and receives a fresh response) per request.
fn serve_responses(listener: std::net::TcpListener, responses: Vec<String>) {
use std::io::{Read, Write};
std::thread::spawn(move || {
for (served, mut stream) in listener.incoming().flatten().enumerate() {
let index = served.min(responses.len() - 1);
// Drain the request first: closing with unread inbound data
// would send a TCP RST and destroy the response in flight
let mut buf = [0u8; 4096];
loop {
match stream.read(&mut buf) {
Ok(0) => break,
Ok(n) if buf[..n].windows(4).any(|w| w == b"\r\n\r\n") => break,
Ok(_) => continue,
Err(_) => break,
}
}
let body = &responses[index];
let _ = stream.write_all(
format!(
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len()
)
.as_bytes(),
);
let _ = stream.flush();
}
});
}
/// One `getPublishedSources` page carrying `versions`, with the
/// `next_collection_link` of a truncated page when `next` is given
fn collection_body(versions: &[&str], next: Option<&str>) -> String {
let entries: Vec<String> = versions
.iter()
.map(|v| format!(r#"{{"source_package_version": "{v}"}}"#))
.collect();
let next_field = next
.map(|link| format!(r#", "next_collection_link": "{link}""#))
.unwrap_or_default();
format!(
r#"{{"start": 0, "total_size": {}, "entries": [{}]{next_field}}}"#,
versions.len(),
entries.join(", ")
)
}
/// Bind a fresh mock server ready to serve `responses` (the caller
/// needs the address to build self-referential `next_collection_link`s
/// before serving starts)
fn bound_collection_server() -> (std::net::TcpListener, String) {
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let base = format!("http://{}", listener.local_addr().unwrap());
(listener, base)
}
/// The collection walk follows `next_collection_link`: the versions of
/// every page are collected in order, and the walk stops on the page
/// without a next link (the mock repeats its last response forever, so
/// an extra fetch would still pass — but a missing next-link handling
/// would drop page two's versions from the result)
#[tokio::test]
async fn walk_collection_collects_every_page() {
let (listener, base) = bound_collection_server();
serve_responses(
listener,
vec![
collection_body(&["1.0-1", "1.6-1"], Some(&format!("{base}/next"))),
collection_body(&["0.9-1"], None),
],
);
let versions = walk_collection(
crate::distro_info::http_client(),
&format!("{base}/~u/+archive/ubuntu/p?ws.op=getPublishedSources"),
"u/p",
)
.await
.unwrap();
assert_eq!(versions, vec!["1.0-1", "1.6-1", "0.9-1"]);
}
/// A next-link chain that never ends must error, not loop forever: the
/// partial list would feed the superseded check a false "not superseded"
#[tokio::test]
async fn walk_collection_errors_when_pagination_never_ends() {
// The last (only) response repeats forever, each page linking back
// to the server: the walk must stop at the page cap by itself
let (listener, base) = bound_collection_server();
serve_responses(
listener,
vec![collection_body(&["1.0-1"], Some(&format!("{base}/loop")))],
);
let err = walk_collection(
crate::distro_info::http_client(),
&format!("{base}/~u/+archive/ubuntu/p?ws.op=getPublishedSources"),
"u/p",
)
.await
.unwrap_err()
.to_string();
assert!(
err.contains("keeps paginating the published sources of 'u/p'"),
"unexpected: {err}"
);
}
}
+27
View File
@@ -9,6 +9,9 @@ pub mod apt;
pub mod build;
/// Parse or edit a Debian changelog of a source package
pub mod changelog;
/// Embedding convention for static reference data (`data/*.yml`), applied
/// by each owning module via the `embed_data!` macro
pub(crate) mod data;
/// Build a Debian package into a binary (.deb)
pub mod deb;
/// Reusable Debian format primitives (control/deb822, checksums, versions,
@@ -16,21 +19,45 @@ pub mod deb;
pub mod debian;
/// Obtain general information about distribution, series, etc
pub mod distro_info;
/// Passive interrupt state: the interrupted flag, the cleanup hook registry
/// and the live view's reporter slot (the CLI owns the signal handling)
pub mod interrupt;
/// Launchpad integration: PPA upload targets and account discovery
pub mod launchpad;
/// Lint a source tree: lintian wrapper for full parity plus pkh-native checks (`pkh lint`)
pub mod lint;
/// Scaffold a new Debian source package (`pkh new`)
pub mod new;
/// Obtain information about one or multiple packages
pub mod package_info;
/// Prune residual pkh build artifacts and caches
pub mod prune;
/// Download a source package locally
pub mod pull;
/// Upload a built source package to a PPA (or archive)
pub mod put;
/// Handle package-specific quirks and workarounds
pub mod quirks;
/// Line classifiers rewriting raw subprocess output into display actions
/// and countable progress (pure logic, shared by build views)
pub mod logfmt;
/// Reporting ports: environment-agnostic build observation ([`BuildView`])
/// and question answering ([`Prompter`]), implemented by terminal views,
/// server bridges or the inert [`Quiet`]
pub mod report;
/// Terminal UI helpers (progress bars, live build views, prompts)
pub mod ui;
/// Handle context for .deb building: locally, over ssh, in a chroot...
pub mod context;
/// Quiet test runs: per-test log files, subprocess capture and failure
/// matrix (inert passthrough outside test binaries)
pub(crate) mod test_support;
/// Utility functions
pub(crate) mod utils;
+106
View File
@@ -0,0 +1,106 @@
//! The check trait and the static check registry.
//!
//! Checks are zero-sized structs, one module per packaging area, registered
//! in one explicit array — reviewable, greppable, and free of proc macros.
//! A registry test fails the build on duplicate tag names, undocumented
//! tags or casing drift, so the catalog cannot rot silently.
use crate::lint::collect::LintData;
use crate::lint::emit::Emitter;
use crate::lint::tag::Tag;
/// A group of checks over one area of the packaging (control, changelog,
/// git workflow, ...). One check instance may emit any of the tags it
/// declares; the emitter resolves metadata and applies suppression.
pub trait Check: Sync {
/// Registry identifier, also the `--check` value (one word, e.g. `pkh-git`).
fn id(&self) -> &'static str;
/// Static metadata of every tag this check may emit.
fn tags(&self) -> &'static [Tag];
/// Run against the collected package information, reporting findings
/// through `emit`.
fn run(&self, data: &LintData, emit: &mut Emitter);
}
/// Every registered check, in catalog order.
pub static CHECKS: &[&dyn Check] = &[&super::checks::pkh::PkhGit as &dyn Check];
/// Look up a registered check by its id (`--check` value).
pub fn find_check(id: &str) -> Option<&'static dyn Check> {
CHECKS.iter().copied().find(|check| check.id() == id)
}
/// Look up tag metadata by tag name across the whole registry.
pub fn find_tag(name: &str) -> Option<&'static Tag> {
CHECKS
.iter()
.flat_map(|check| check.tags())
.find(|tag| tag.name == name)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn registry_tag_names_are_unique_and_well_formed() {
let mut seen = Vec::new();
for tag in CHECKS.iter().flat_map(|check| check.tags()) {
assert!(
!seen.contains(&tag.name),
"duplicate tag name: {}",
tag.name
);
seen.push(tag.name);
let valid = |c: char| c.is_ascii_lowercase() || c.is_ascii_digit() || "+.-".contains(c);
assert!(
tag.name.starts_with(valid)
&& tag.name.chars().all(valid)
&& !tag.name.ends_with('-'),
"tag name is not kebab-case: {}",
tag.name
);
assert!(
!tag.description.trim().is_empty(),
"tag without a description: {}",
tag.name
);
}
}
#[test]
fn check_ids_are_unique_and_resolvable() {
let mut seen = Vec::new();
for check in CHECKS {
assert!(
!seen.contains(&check.id()),
"duplicate check id: {}",
check.id()
);
seen.push(check.id());
assert!(
find_check(check.id()).is_some(),
"find_check cannot resolve its own registry: {}",
check.id()
);
}
}
#[test]
fn pkh_native_tags_are_namespaced() {
for check in CHECKS {
if check.id().starts_with("pkh-") {
for tag in check.tags() {
assert!(
tag.name.starts_with("pkh-"),
"pkh-native check '{}' emits non-namespaced tag '{}'",
check.id(),
tag.name
);
}
}
}
}
}
+4
View File
@@ -0,0 +1,4 @@
//! Check areas; one module per area, registered in [`crate::lint::check::CHECKS`].
/// Pkh-native workflow checks (git-centric trees, PPA uploads).
pub mod pkh;
+148
View File
@@ -0,0 +1,148 @@
//! Pkh-native checks: workflow knowledge lintian cannot have, because it
//! lives in pkh's flows (git-centric trees, PPA uploads, scaffolding).
use crate::lint::check::Check;
use crate::lint::collect::LintData;
use crate::lint::emit::Emitter;
use crate::lint::tag::{Certainty, Severity, Tag};
/// Metadata of every tag the pkh-native checks emit.
pub static TAGS: &[Tag] = &[Tag {
name: "pkh-debian-changes-not-committed",
severity: Severity::Warning,
certainty: Certainty::Certain,
experimental: false,
description: "The debian/ directory contains changes that are not committed to git. \
pkh builds and uploads the tree as-is (pkh deb, pkh put); committing first keeps \
the upload and the git history in sync.",
references: &[],
}];
/// Flags `debian/` content that exists in the tree but is not committed to
/// git. Skips trees outside any git repository: archive-pulled sources
/// legitimately have none.
pub struct PkhGit;
impl Check for PkhGit {
fn id(&self) -> &'static str {
"pkh-git"
}
fn tags(&self) -> &'static [Tag] {
TAGS
}
fn run(&self, data: &LintData, emit: &mut Emitter) {
let Some(git) = &data.git else {
return;
};
if git.dirty_debian.is_empty() {
return;
}
let examples: Vec<&str> = git
.dirty_debian
.iter()
.take(3)
.map(String::as_str)
.collect();
let more = if git.dirty_debian.len() > examples.len() {
", ..."
} else {
""
};
emit.tag(
"pkh-debian-changes-not-committed",
format!(
"{} uncommitted change(s) under debian/ (e.g. {}{more})",
git.dirty_debian.len(),
examples.join(", ")
),
);
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::Path;
fn write(path: &Path, content: &str) {
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(path, content).unwrap();
}
fn commit_all(repo: &git2::Repository) {
let signature = git2::Signature::now("pkh test", "test@example.com").unwrap();
let mut index = repo.index().unwrap();
index
.add_all(["*"], git2::IndexAddOption::DEFAULT, None)
.unwrap();
// write_tree alone does not persist the index; without this, a
// committed worktree still reads as index-deleted + untracked.
index.write().unwrap();
let tree_id = index.write_tree().unwrap();
let tree = repo.find_tree(tree_id).unwrap();
repo.commit(Some("HEAD"), &signature, &signature, "init", &tree, &[])
.unwrap();
}
#[test]
fn no_git_repository_is_no_finding() {
let dir = tempfile::tempdir().unwrap();
write(&dir.path().join("debian/control"), "Source: hello\n");
let data = LintData::collect(dir.path(), None, None);
assert!(data.git.is_none());
let mut findings = Vec::new();
let mut emitter = Emitter::new(data.source_name.clone(), &mut findings);
PkhGit.run(&data, &mut emitter);
assert!(findings.is_empty());
}
#[test]
fn clean_worktree_is_no_finding() {
let dir = tempfile::tempdir().unwrap();
write(&dir.path().join("debian/control"), "Source: hello\n");
let repo = git2::Repository::init(dir.path()).unwrap();
commit_all(&repo);
let data = LintData::collect(dir.path(), None, None);
let mut findings = Vec::new();
let mut emitter = Emitter::new(data.source_name.clone(), &mut findings);
PkhGit.run(&data, &mut emitter);
assert!(findings.is_empty());
}
#[test]
fn dirty_debian_tree_is_a_finding() {
let dir = tempfile::tempdir().unwrap();
write(&dir.path().join("debian/control"), "Source: hello\n");
write(&dir.path().join("hello.txt"), "upstream\n");
let repo = git2::Repository::init(dir.path()).unwrap();
commit_all(&repo);
// A modified tracked file and a fresh untracked patch: both count.
write(
&dir.path().join("debian/control"),
"Source: hello\nDepends: x\n",
);
write(&dir.path().join("debian/patches/new.patch"), "...\n");
write(&dir.path().join("hello.txt"), "changed upstream\n");
let data = LintData::collect(dir.path(), None, None);
let mut findings = Vec::new();
let mut emitter = Emitter::new(data.source_name.clone(), &mut findings);
PkhGit.run(&data, &mut emitter);
assert_eq!(findings.len(), 1);
let finding = &findings[0];
assert_eq!(finding.tag_name, "pkh-debian-changes-not-committed");
assert_eq!(finding.letter, 'W');
assert!(
finding
.message
.starts_with("2 uncommitted change(s) under debian/")
);
assert!(finding.message.contains("debian/control"));
assert!(finding.message.contains("debian/patches/new.patch"));
}
}
+98
View File
@@ -0,0 +1,98 @@
//! Collectors: package information gathered once per run and shared by every
//! check, mirroring lintian's collection phase without the on-disk lab.
//! Source trees are small, so everything is computed eagerly except git
//! status, which only exists when the tree is a git worktree.
use std::path::{Path, PathBuf};
use crate::debian::control::ControlInfo;
/// Git worktree state relevant to pkh's git-centric workflow checks.
pub struct GitStatus {
/// Repo-relative paths under `debian/` with uncommitted content
/// (modified, staged or untracked), sorted.
pub dirty_debian: Vec<String>,
}
/// Everything the checks and the report renderer know about the linted tree.
pub struct LintData {
/// Root of the source tree being linted.
pub root: PathBuf,
/// Source package name, resolved from `debian/control`, else from the
/// changelog's first line, else the directory name.
pub source_name: String,
/// Target distribution (`--dist`); unresolved when None.
pub dist: Option<String>,
/// Target series (`--series`); unresolved when None.
pub series: Option<String>,
/// Git worktree state; None when the tree is not inside a git repository.
pub git: Option<GitStatus>,
}
impl LintData {
/// Collect information about the source tree at `root`.
pub fn collect(root: &Path, dist: Option<&str>, series: Option<&str>) -> LintData {
LintData {
root: root.to_path_buf(),
source_name: resolve_source_name(root),
dist: dist.map(str::to_string),
series: series.map(str::to_string),
git: collect_git(root),
}
}
}
/// Source package name from `debian/control`, falling back to the changelog
/// header and then the directory name; the report needs a display name even
/// for broken trees.
fn resolve_source_name(root: &Path) -> String {
if let Ok(control) = ControlInfo::parse(&root.join("debian/control")) {
return control.source_name().to_string();
}
if let Ok(changelog) = std::fs::read_to_string(root.join("debian/changelog"))
&& let Some(first) = changelog.lines().next()
&& let Some(name) = first.split(" (").next()
&& !name.trim().is_empty()
{
return name.trim().to_string();
}
root.file_name()
.map(|name| name.to_string_lossy().to_string())
.unwrap_or_else(|| "package".to_string())
}
/// Dirty paths under `debian/` when the tree lives in a git worktree.
/// Untracked files count: a fresh patch nobody committed is exactly the
/// mistake the workflow checks exist to catch. None when there is no
/// repository (or git is somehow unusable here) — never a finding, since
/// archive-pulled trees legitimately have none.
fn collect_git(root: &Path) -> Option<GitStatus> {
let repo = git2::Repository::discover(root).ok()?;
let workdir = repo.workdir()?;
// Statuses are workdir-relative; trees nested inside a repository only
// care about their own slice of it.
let prefix = root.strip_prefix(workdir).unwrap_or(Path::new(""));
let debian_dir = prefix.join("debian");
let mut options = git2::StatusOptions::new();
options
.include_untracked(true)
.include_ignored(false)
.recurse_untracked_dirs(true);
let statuses = repo.statuses(Some(&mut options)).ok()?;
let mut dirty_debian = Vec::new();
for entry in statuses.iter() {
// CURRENT is the zero flag in libgit2, so any non-empty status is
// some kind of change (worktree or staged).
if entry.status().is_empty() {
continue;
}
let path = entry.path()?;
if Path::new(path).starts_with(&debian_dir) {
dirty_debian.push(path.to_string());
}
}
dirty_debian.sort();
Some(GitStatus { dirty_debian })
}
+118
View File
@@ -0,0 +1,118 @@
//! Findings, the run report, and the emitter checks report through.
//!
//! The emitter is deliberately thin: checks name a tag and give a message;
//! metadata, output letter and explanations come from the registry, so
//! findings from the native engine and findings parsed from the wrapped
//! lintian share one shape and one rendering path.
use crate::lint::check;
use crate::lint::tag::Tag;
/// Where a finding comes from.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Origin {
/// Emitted by pkh's native Rust checks.
Native,
/// Parsed from the wrapped lintian's output.
Lintian,
}
/// One lint finding, whatever produced it.
#[derive(Debug, Clone)]
pub struct Finding {
/// Output letter (`E`, `W`, `I`, `P`, `X`, `O`, `C`): severity for
/// native findings, verbatim from the output line in wrapper mode.
pub letter: char,
/// Stable tag name (`missing-debian-copyright-file`).
pub tag_name: String,
/// Free-form details after the tag name; empty when the tag stands alone.
pub message: String,
/// Package the finding belongs to (`hello`).
pub package: String,
/// Processable type lintian displays after the package name (`source`,
/// `changes`, ...); None for binary findings, which lintian prints
/// without a type.
pub processable_type: Option<String>,
/// Explanation lines shown by `--info`: the `N:` blocks lintian attaches
/// to the finding in wrapper mode, or the native tag's description and
/// references.
pub explanation: Vec<String>,
/// Native or parsed-from-lintian.
pub origin: Origin,
}
impl Finding {
/// Lowercase severity/classification name for this finding's letter, as
/// used in `--fail-on` values and JSON output.
pub fn severity_name(&self) -> &'static str {
match self.letter {
'E' => "error",
'W' => "warning",
'I' => "info",
'P' => "pedantic",
'X' => "experimental",
'O' => "overridden",
_ => "classification",
}
}
}
/// The full result of one lint run: every finding, from every source.
pub struct LintReport {
/// All findings in emission order: the wrapper's first, then the native
/// ones (deduplicated by tag name against the wrapper's).
pub findings: Vec<Finding>,
/// Display name of the linted source package.
pub source_name: String,
/// The wrapper was skipped because lintian is not installed; the
/// renderer prints a notice and the run is native-only.
pub wrapper_unavailable: bool,
/// Run-level notes (`N:` lines): how the linted artifact was obtained,
/// and similar context that is not a finding.
pub notes: Vec<String>,
}
/// Sink checks report findings through. Resolves tag metadata from the
/// registry so checks only ever name the tag they mean.
pub struct Emitter<'a> {
package: String,
findings: &'a mut Vec<Finding>,
}
impl<'a> Emitter<'a> {
/// Emitter for findings of `package` (e.g. the source package name),
/// appending into `findings`.
pub fn new(package: String, findings: &'a mut Vec<Finding>) -> Emitter<'a> {
Emitter { package, findings }
}
/// Emit `tag_name` with `message` as its detail line. Native findings
/// always report on the source package; an unknown tag name (a registry
/// bug) still produces a finding rather than panicking — the registry
/// test makes that path unreachable in practice.
pub fn tag(&mut self, tag_name: &str, message: impl Into<String>) {
let (letter, explanation) = match check::find_tag(tag_name) {
Some(tag) => (tag.letter(), explanation_of(tag)),
None => ('E', Vec::new()),
};
self.findings.push(Finding {
letter,
tag_name: tag_name.to_string(),
message: message.into(),
package: self.package.clone(),
processable_type: Some("source".to_string()),
explanation,
origin: Origin::Native,
});
}
}
/// The explanation `--info` shows for a native tag: its description plus any
/// references, prefixed like lintian's "Please refer to" pointers.
fn explanation_of(tag: &Tag) -> Vec<String> {
let mut lines = vec![tag.description.to_string()];
for reference in tag.references {
lines.push(format!("Please refer to {}", reference));
}
lines
}
+186
View File
@@ -0,0 +1,186 @@
//! `pkh lint`: lint a Debian source tree, with lintian feature parity.
//!
//! The strategy is *wrap first, port second* (see `plans/pkh-lint.md`): the
//! [wrapper](wrapper) runs the installed lintian over an ephemeral source
//! package for day-one parity with every lintian check, while the native
//! engine (check registry, collectors, emitter) hosts pkh-specific workflow
//! checks (`pkh-*` tags) and grows ported lintian checks incrementally.
//! Both findings merge into one report — deduplicated by tag name, which is
//! why native checks mirror lintian's tag names for equivalent checks —
//! rendered as lintian-shaped text or JSON, with lintian's exit-code
//! contract (0 clean, 1 findings at/above `--fail-on`, 2 runtime error).
pub mod check;
pub mod checks;
pub mod collect;
pub mod emit;
pub mod output;
pub mod tag;
pub mod wrapper;
use std::collections::HashSet;
use std::path::PathBuf;
use crate::lint::check::Check;
use crate::lint::collect::LintData;
use crate::lint::emit::{Finding, LintReport};
/// Knobs of one `pkh lint` run, built from the CLI in `main.rs`.
pub struct LintOptions {
/// Source tree to lint.
pub path: PathBuf,
/// Run the native engine only; never invoke the lintian wrapper.
pub native: bool,
/// Levels that make the exit code 1 (`--fail-on`, default: error).
pub fail_on: Vec<output::Level>,
/// Show tag explanations under each finding (`--info`, lintian's `-i`).
pub info: bool,
/// Also display info-level (`I:`) findings (lintian's `-I`).
pub display_info: bool,
/// Also display pedantic (`P:`) findings.
pub pedantic: bool,
/// Also display experimental (`X:`) findings.
pub experimental: bool,
/// Also display overridden (`O:`) findings.
pub show_overrides: bool,
/// Tag names to ignore for this run.
pub suppress_tags: Vec<String>,
/// Run only these native checks (`--check`, repeatable).
pub only_checks: Vec<String>,
/// Ignore any existing `pkh build` output and pack the tree fresh.
pub repack: bool,
/// Emit JSON instead of text.
pub json: bool,
/// Text colorization mode (`--color`, default auto: TTY without NO_COLOR).
pub color: output::ColorMode,
/// Target distribution (debian/ubuntu), when known.
pub dist: Option<String>,
/// Target series, when known.
pub series: Option<String>,
}
/// Run one lint: collect package information, run the native checks, wrap
/// lintian (unless `--native`), and merge everything into one report.
pub fn run(options: &LintOptions) -> Result<LintReport, String> {
let data = LintData::collect(
&options.path,
options.dist.as_deref(),
options.series.as_deref(),
);
let mut report = LintReport {
findings: Vec::new(),
source_name: data.source_name.clone(),
wrapper_unavailable: false,
notes: Vec::new(),
};
if !options.native {
match wrapper::run(&data.root, options)? {
Some(outcome) => {
report.findings.extend(outcome.findings);
report.notes = outcome.notes;
}
None => report.wrapper_unavailable = true,
}
}
let selected: Vec<&'static dyn Check> = if options.only_checks.is_empty() {
check::CHECKS.to_vec()
} else {
options
.only_checks
.iter()
.map(|id| check::find_check(id).ok_or_else(|| format!("Unknown --check '{id}'")))
.collect::<Result<_, _>>()?
};
let mut native = Vec::new();
{
let mut emitter = emit::Emitter::new(report.source_name.clone(), &mut native);
for check in selected {
check.run(&data, &mut emitter);
}
}
report.findings.extend(merge(&report.findings, native));
if !options.suppress_tags.is_empty() {
report
.findings
.retain(|finding| !options.suppress_tags.contains(&finding.tag_name));
}
Ok(report)
}
/// Append `native` findings to `lintian`'s, dropping native duplicates by
/// tag name (the wrapper's verdict wins for tags both engines produce).
fn merge(lintian: &[Finding], native: Vec<Finding>) -> Vec<Finding> {
let known: HashSet<&str> = lintian.iter().map(|f| f.tag_name.as_str()).collect();
native
.into_iter()
.filter(|finding| !known.contains(finding.tag_name.as_str()))
.collect()
}
/// The `--list-tags` catalog: one line per registered native tag.
pub fn list_tags() -> String {
let mut out = String::new();
for check in check::CHECKS {
for tag in check.tags() {
out.push_str(&format!(
"{} [{}] {}\n {}\n",
tag.letter(),
check.id(),
tag.name,
tag.description
));
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use crate::lint::emit::Origin;
fn finding(tag: &str, origin: Origin) -> Finding {
Finding {
letter: 'W',
tag_name: tag.to_string(),
message: String::new(),
package: "hello".to_string(),
processable_type: Some("source".to_string()),
explanation: Vec::new(),
origin,
}
}
#[test]
fn merge_drops_native_duplicates_by_tag_name() {
let lintian = vec![finding("no-debian-copyright-in-source", Origin::Lintian)];
let native = vec![
finding("no-debian-copyright-in-source", Origin::Native),
finding("pkh-debian-changes-not-committed", Origin::Native),
];
let merged = merge(&lintian, native);
assert_eq!(merged.len(), 1);
assert_eq!(merged[0].tag_name, "pkh-debian-changes-not-committed");
assert_eq!(merged[0].origin, Origin::Native);
}
#[test]
fn native_tags_resolve_from_the_registry() {
let tag = check::find_tag("pkh-debian-changes-not-committed")
.expect("pkh check tags must resolve");
assert_eq!(tag.letter(), 'W');
}
#[test]
fn unknown_native_tags_still_produce_findings() {
let mut sink = Vec::new();
let mut emitter = emit::Emitter::new("hello".to_string(), &mut sink);
emitter.tag("no-such-tag-anywhere", "boom");
assert_eq!(sink.len(), 1);
assert_eq!(sink[0].tag_name, "no-such-tag-anywhere");
}
}
+450
View File
@@ -0,0 +1,450 @@
//! Rendering of a lint report (text and JSON) and the exit-code decision.
//!
//! Text output keeps lintian's line shape (`<L>: <pkg> <type>: <tag>
//! <details>`) so findings read identically in both modes; JSON is the
//! schema-stable form for CI. Display filtering (what is shown) and the
//! `--fail-on` threshold (what makes the exit code 1) are independent, like
//! lintian's. Colorization happens here, at render time — findings are
//! captured plain (`--color never` is passed to lintian) and re-painted by
//! severity, so colors are a renderer concern that survives the switch from
//! wrapped lintian output to native checks.
use crossterm::style::Stylize;
use serde_json::json;
use crate::lint::LintOptions;
use crate::lint::emit::{Finding, LintReport, Origin};
/// When to colorize the text report.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum ColorMode {
/// Colorize when stdout is a terminal and `NO_COLOR` is unset.
#[default]
Auto,
/// Always colorize (piping, CI logs).
Always,
/// Never colorize.
Never,
}
impl ColorMode {
/// Parse one `--color` keyword.
pub fn parse(word: &str) -> Option<ColorMode> {
match word {
"auto" => Some(ColorMode::Auto),
"always" => Some(ColorMode::Always),
"never" => Some(ColorMode::Never),
_ => None,
}
}
/// Whether the text renderer should emit ANSI colors.
fn should_color(self) -> bool {
match self {
ColorMode::Always => true,
ColorMode::Never => false,
ColorMode::Auto => {
std::io::IsTerminal::is_terminal(&std::io::stdout())
&& std::env::var_os("NO_COLOR").is_none()
}
}
}
}
/// The output letter, painted with its severity color when `color` is set:
/// errors red (bold), warnings yellow, info cyan, pedantic/experimental
/// magenta, overridden green. Mirrors lintian's tty palette closely enough
/// for muscle memory.
fn paint_letter(letter: char, color: bool) -> String {
if !color {
return letter.to_string();
}
match letter {
'E' => "E".red().bold().to_string(),
'W' => "W".yellow().to_string(),
'I' => "I".cyan().to_string(),
'P' => "P".magenta().to_string(),
'X' => "X".magenta().to_string(),
'O' => "O".green().to_string(),
_ => letter.to_string(),
}
}
/// A named severity class, mirroring the values lintian's `--fail-on`
/// accepts. `Experimental` is the `X:` pseudo-level (the letter hides the
/// underlying severity) and `Overridden` lets gates count suppressed tags.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Level {
/// `E:` findings.
Error,
/// `W:` findings.
Warning,
/// `I:` findings.
Info,
/// `P:` findings.
Pedantic,
/// `X:` findings.
Experimental,
/// `O:` findings (only reported when overrides are shown).
Overridden,
}
impl Level {
/// The output letter the level stands for.
pub fn letter(self) -> char {
match self {
Level::Error => 'E',
Level::Warning => 'W',
Level::Info => 'I',
Level::Pedantic => 'P',
Level::Experimental => 'X',
Level::Overridden => 'O',
}
}
/// Parse one comma-separated `--fail-on` keyword.
pub fn parse(word: &str) -> Option<Level> {
match word.trim() {
"error" => Some(Level::Error),
"warning" => Some(Level::Warning),
"info" => Some(Level::Info),
"pedantic" => Some(Level::Pedantic),
"experimental" => Some(Level::Experimental),
"override" => Some(Level::Overridden),
_ => None,
}
}
}
/// Parse the whole `--fail-on` value (comma-separated level names).
pub fn parse_fail_on(spec: &str) -> Result<Vec<Level>, String> {
let levels: Vec<Level> = spec
.split(',')
.filter(|word| !word.trim().is_empty())
.map(Level::parse)
.collect::<Option<_>>()
.ok_or_else(|| {
format!(
"Invalid --fail-on value '{spec}': expected comma-separated \
error, warning, info, pedantic, experimental or override"
)
})?;
if levels.is_empty() {
return Err("Empty --fail-on value: expected at least one level".to_string());
}
Ok(levels)
}
/// The letter a finding displays as for display-level purposes: overridden
/// findings carry their `O:` letter, classification tags are JSON-only.
fn is_displayed(finding: &Finding, options: &LintOptions) -> bool {
match finding.letter {
'E' | 'W' => true,
'I' => options.display_info,
'P' => options.pedantic,
'X' => options.experimental,
'O' => options.show_overrides,
_ => false,
}
}
/// Exit code of the run: 1 when any finding (except classification tags) is
/// at a level listed in `--fail-on`, 0 otherwise. Runtime failures never
/// reach this function — the caller exits 2 directly.
pub fn exit_code(report: &LintReport, options: &LintOptions) -> i32 {
let failed = report
.findings
.iter()
.any(|finding| match level_of_letter(finding.letter) {
Some(level) => options.fail_on.contains(&level),
None => false,
});
if failed { 1 } else { 0 }
}
/// The level a finding's letter maps to for `--fail-on` purposes; None for
/// classification tags, which lintian also never fails on.
fn level_of_letter(letter: char) -> Option<Level> {
match letter {
'E' => Some(Level::Error),
'W' => Some(Level::Warning),
'I' => Some(Level::Info),
'P' => Some(Level::Pedantic),
'X' => Some(Level::Experimental),
'O' => Some(Level::Overridden),
_ => None,
}
}
/// Render the report as lintian-shaped text: one `<L>: <pkg> <type>: <tag>
/// <details>` line per displayed finding, explanations under `--info`, then
/// an `N:` summary line.
pub fn render_text(report: &LintReport, options: &LintOptions) -> String {
let color = options.color.should_color();
let mut out = String::new();
if report.wrapper_unavailable {
out.push_str(
"N: lintian is not installed; showing pkh-native checks only \
(install lintian for full check coverage)\n",
);
}
for note in &report.notes {
out.push_str(&format!("N: {note}\n"));
}
let mut shown_counts = [('E', 0), ('W', 0), ('I', 0), ('P', 0), ('X', 0), ('O', 0)];
let mut shown = 0;
for finding in &report.findings {
if !is_displayed(finding, options) {
continue;
}
shown += 1;
if let Some((_, count)) = shown_counts
.iter_mut()
.find(|(letter, _)| *letter == finding.letter)
{
*count += 1;
}
out.push_str(&format!(
"{}: {}\n",
paint_letter(finding.letter, color),
line_subject(finding)
));
if options.info {
for line in &finding.explanation {
if line.is_empty() {
out.push_str("N:\n");
} else {
out.push_str(&format!("N: {}\n", line.trim_end()));
}
}
}
}
let mut parts = Vec::new();
for (letter, count) in shown_counts {
if count > 0 {
parts.push(format!("{} {}", count, paint_letter(letter, color)));
}
}
let hidden = report.findings.len() - shown;
if parts.is_empty() {
out.push_str(&format!("N: no displayed tags; {hidden} hidden\n"));
} else {
out.push_str(&format!(
"N: {shown} tag(s) shown ({}); {hidden} hidden\n",
parts.join(", ")
));
}
out
}
/// `hello source: tag details` — the part of a lintian line after the
/// letter, reproduced identically for both origins.
fn line_subject(finding: &Finding) -> String {
let mut line = String::from(&finding.package);
if let Some(ptype) = &finding.processable_type {
line.push(' ');
line.push_str(ptype);
}
line.push_str(": ");
line.push_str(&finding.tag_name);
if !finding.message.is_empty() {
line.push(' ');
line.push_str(&finding.message);
}
line
}
/// Render the report as pretty-printed JSON: every finding (displayed or
/// not, flagged as such) plus a summary carrying the exit-code verdict.
pub fn render_json(report: &LintReport, options: &LintOptions) -> String {
let findings: Vec<serde_json::Value> = report
.findings
.iter()
.map(|finding| {
json!({
"letter": finding.letter.to_string(),
"tag": finding.tag_name,
"severity": finding.severity_name(),
"package": finding.package,
"processable_type": finding.processable_type,
"message": finding.message,
"origin": match finding.origin {
Origin::Native => "native",
Origin::Lintian => "lintian",
},
"overridden": finding.letter == 'O',
"displayed": is_displayed(finding, options),
"explanation": if finding.explanation.is_empty() {
json!(null)
} else {
json!(finding.explanation)
},
})
})
.collect();
let document = json!({
"source": report.source_name,
"wrapper_unavailable": report.wrapper_unavailable,
"notes": report.notes,
"findings": findings,
"summary": {
"failed": exit_code(report, options) == 1,
},
});
serde_json::to_string_pretty(&document).expect("lint report JSON is serializable")
}
#[cfg(test)]
mod tests {
use super::*;
fn options(fail_on: &[Level]) -> LintOptions {
LintOptions {
path: std::path::PathBuf::from("."),
native: false,
fail_on: fail_on.to_vec(),
info: false,
display_info: false,
pedantic: false,
experimental: false,
show_overrides: false,
suppress_tags: Vec::new(),
only_checks: Vec::new(),
repack: false,
json: false,
color: ColorMode::Never,
dist: None,
series: None,
}
}
fn report(findings: &[(&str, char)]) -> LintReport {
LintReport {
findings: findings
.iter()
.map(|(tag, letter)| Finding {
letter: *letter,
tag_name: (*tag).to_string(),
message: String::new(),
package: "hello".to_string(),
processable_type: Some("source".to_string()),
explanation: Vec::new(),
origin: Origin::Lintian,
})
.collect(),
source_name: "hello".to_string(),
wrapper_unavailable: false,
notes: Vec::new(),
}
}
#[test]
fn fail_on_parses_and_rejects_unknown_levels() {
assert_eq!(
parse_fail_on("error, warning").unwrap(),
vec![Level::Error, Level::Warning]
);
assert!(parse_fail_on("bogus").is_err());
assert!(parse_fail_on(" ").is_err());
}
#[test]
fn exit_code_only_counts_displayed_severity_letters() {
let default_opts = options(&[Level::Error]);
// Errors fail, warnings alone do not (the default threshold).
assert_eq!(exit_code(&report(&[("t", 'E')]), &default_opts), 1);
assert_eq!(
exit_code(&report(&[("t", 'W'), ("u", 'I')]), &default_opts),
0
);
// Overridden findings never fail unless explicitly requested.
assert_eq!(exit_code(&report(&[("t", 'O')]), &default_opts), 0);
assert_eq!(
exit_code(
&report(&[("t", 'O')]),
&options(&[Level::Error, Level::Overridden])
),
1
);
// Classification tags never fail.
assert_eq!(exit_code(&report(&[("t", 'C')]), &default_opts), 0);
}
#[test]
fn text_rendering_respects_display_levels() {
let base = report(&[("e", 'E'), ("i", 'I'), ("o", 'O'), ("x", 'X')]);
let default = render_text(&base, &options(&[Level::Error]));
assert!(default.contains("E: hello source: e"));
assert!(!default.contains("I: "));
assert!(!default.contains("O: "));
assert!(!default.contains("X: "));
assert!(default.ends_with("N: 1 tag(s) shown (1 E); 3 hidden\n"));
let everything = LintOptions {
display_info: true,
experimental: true,
show_overrides: true,
..options(&[Level::Error])
};
let full = render_text(&base, &everything);
for prefix in ["E:", "I:", "O:", "X:"] {
assert!(
full.contains(&format!("{prefix} hello source:")),
"{prefix}"
);
}
assert!(full.ends_with("N: 4 tag(s) shown (1 E, 1 I, 1 X, 1 O); 0 hidden\n"));
}
#[test]
fn colorization_is_render_only_and_mode_switchable() {
let run_report = report(&[("e", 'E'), ("w", 'W'), ("o", 'O')]);
let plain = options(&[Level::Error]);
let mut forced = options(&[Level::Error]);
forced.color = ColorMode::Always;
let plain = render_text(&run_report, &plain);
let colored = render_text(&run_report, &forced);
assert!(
!plain.contains('\u{1b}'),
"never/auto-on-pipe must stay plain"
);
assert!(colored.contains('\u{1b}'), "always must colorize");
// Colors wrap the letters only; the lintian line shape is intact.
assert!(colored.contains("hello source: e"));
// Mode parsing round-trips.
assert_eq!(ColorMode::parse("always"), Some(ColorMode::Always));
assert_eq!(ColorMode::parse("bogus"), None);
}
#[test]
fn json_round_trips_with_verdicts() {
let run_report = report(&[("e", 'E'), ("w", 'W')]);
let default_opts = options(&[Level::Error]);
let document: serde_json::Value =
serde_json::from_str(&render_json(&run_report, &default_opts)).unwrap();
assert_eq!(document["source"], "hello");
assert_eq!(document["findings"][0]["letter"], "E");
assert_eq!(document["findings"][0]["severity"], "error");
assert_eq!(document["findings"][0]["origin"], "lintian");
assert_eq!(document["summary"]["failed"], true);
// An error finding fails at the warning threshold too, but not at
// pedantic-only, which nothing in this report reaches.
let warn_only = options(&[Level::Warning]);
assert_eq!(exit_code(&run_report, &warn_only), 1);
let document: serde_json::Value =
serde_json::from_str(&render_json(&run_report, &warn_only)).unwrap();
assert_eq!(document["summary"]["failed"], true);
let pedantic_only = options(&[Level::Pedantic]);
assert_eq!(exit_code(&run_report, &pedantic_only), 0);
let document: serde_json::Value =
serde_json::from_str(&render_json(&run_report, &pedantic_only)).unwrap();
assert_eq!(document["summary"]["failed"], false);
}
}
+85
View File
@@ -0,0 +1,85 @@
//! Lint tag model: severities, certainties and static tag metadata.
//!
//! Tags are the atomic diagnostics of a lint run, named after lintian's
//! model: a stable machine-readable name (`missing-debian-copyright-file`),
//! a severity, a certainty and a description. Checks declare the tags they
//! may emit as static [`Tag`] values; the wrapper's parsed findings carry the
//! letter lintian printed instead.
/// Severity of a finding, matching lintian's severity ladder.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub enum Severity {
/// Policy violation or broken package data (`E:`).
Error,
/// Probable bug or policy deviation (`W:`).
Warning,
/// Informational note about packaging choices (`I:`).
Info,
/// Nitpick most packages may legitimately ignore (`P:`).
Pedantic,
}
impl Severity {
/// The output letter lintian displays this severity as (`E`, `W`, ...).
pub fn letter(self) -> char {
match self {
Severity::Error => 'E',
Severity::Warning => 'W',
Severity::Info => 'I',
Severity::Pedantic => 'P',
}
}
/// Lowercase name used in `--fail-on` values and JSON output.
pub fn name(self) -> &'static str {
match self {
Severity::Error => "error",
Severity::Warning => "warning",
Severity::Info => "info",
Severity::Pedantic => "pedantic",
}
}
}
/// How sure a check is that a finding is real. Not acted upon yet (a future
/// `--fail-on error,certain` would consume it), but captured from day one so
/// severity tuning is data-driven later.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Certainty {
/// The finding is a fact (e.g. a missing file).
Certain,
/// The finding is likely but has legitimate exceptions.
Possible,
/// The finding is a guess from weak signals.
WildGuess,
}
/// Static metadata of one tag: what checks declare and the renderer resolves.
#[derive(Debug)]
pub struct Tag {
/// Stable machine-readable name, lintian-compatible kebab-case
/// (`pkh-debian-changes-not-committed`).
pub name: &'static str,
/// Severity the tag reports at.
pub severity: Severity,
/// How sure checks are when emitting this tag.
pub certainty: Certainty,
/// Whether the tag is experimental (`X:` output, hidden by default).
pub experimental: bool,
/// One-paragraph explanation, shown by `--info` and `--list-tags`.
pub description: &'static str,
/// References (policy sections, URLs) shown by `--info`.
pub references: &'static [&'static str],
}
impl Tag {
/// The output letter for this tag: experimental tags render as `X:`
/// regardless of their severity, like lintian.
pub fn letter(&self) -> char {
if self.experimental {
'X'
} else {
self.severity.letter()
}
}
}
+531
View File
@@ -0,0 +1,531 @@
//! The lintian wrapper: pkh's day-one feature-parity layer.
//!
//! Lintian only accepts built package files, never source trees, so the
//! wrapper needs a source artifact. It lints the `pkh build` output next to
//! the tree when it matches the current changelog entry and nothing in the
//! tree is newer than it (the fast path: no packing at all); otherwise it
//! packs the tree fresh with `dpkg-source -b` inside a temporary directory
//! and lints the resulting `.dsc` (`--repack` forces that path). Nothing is
//! written back into the linted tree; the temp directory is removed on drop.
//!
//! Parsing is load-bearing here (findings are merged with the native ones,
//! rendered uniformly and exported as JSON), so the parser is pinned by
//! golden tests captured from real lintian output. Lintian's own exit code
//! is *not* authoritative: lintian uses 2 both for "fail-on met" and for
//! runtime errors, while pkh derives the verdict from the parsed findings
//! and reserves 2 for actual runtime failures.
use std::io::ErrorKind;
use std::path::{Path, PathBuf};
use std::process::Command;
use std::time::{SystemTime, UNIX_EPOCH};
use regex::Regex;
use crate::lint::LintOptions;
use crate::lint::emit::{Finding, Origin};
/// What one wrapper run produced: the parsed lintian findings plus `N:`
/// notes about how the artifact being linted was obtained.
pub struct WrapperOutcome {
/// Lintian's findings.
pub findings: Vec<Finding>,
/// Human-visible notes (artifact reuse, fresh packing) rendered as `N:`.
pub notes: Vec<String>,
}
/// Run the wrapper over the source tree at `root`:
/// - `Ok(Some(outcome))` — lintian ran; these are its findings and notes,
/// - `Ok(None)` — lintian is not installed; the caller falls back to
/// native-only with a notice,
/// - `Err(message)` — runtime failure (tree unpacked badly, lintian crashed
/// without reportable output); the caller exits 2.
///
/// The artifact linted is the `pkh build` output next to the tree when it
/// matches the current changelog entry and no tree content is newer;
/// otherwise the tree is packed fresh (with `--repack` forcing that path).
pub fn run(root: &Path, options: &LintOptions) -> Result<Option<WrapperOutcome>, String> {
let tmp = TempDir::new()?;
let root = root
.canonicalize()
.map_err(|e| format!("Cannot lint '{}': {e}", root.display()))?;
let (dsc, notes) = match usable_build_output(&root, options.repack) {
Some(dsc) => {
let note = format!(
"linting pkh build output {}",
crate::report::display_path(&dsc)
);
(dsc, vec![note])
}
None => {
let dsc = pack(&root, &tmp)?;
let reason = if options.repack {
"the tree was packed fresh with dpkg-source (--repack ignored \
the existing pkh build output)"
.to_string()
} else if expected_build_output(&root).is_some() {
"the tree changed since pkh build, so it was packed fresh \
with dpkg-source (rerun pkh build to lint the build output)"
.to_string()
} else {
"no pkh build output next to the tree, so it was packed fresh \
with dpkg-source (pkh build produces one)"
.to_string()
};
(dsc, vec![reason])
}
};
let mut lintian = Command::new("lintian");
lintian
.env("LC_ALL", "C")
.args(["--no-cfg", "--color", "never"])
.arg("--info")
.arg(&dsc);
// Per-distro scoping: lintian auto-detects the *host* vendor, but pkh
// knows the *target* distro (-d); make the two agree, which matters on
// cross-distro hosts (linting an Ubuntu package on Debian or back).
if let Some(dist) = &options.dist
&& matches!(dist.as_str(), "ubuntu" | "debian")
{
lintian.arg("--profile").arg(dist);
}
if options.display_info {
lintian.arg("--display-info");
}
if options.pedantic {
lintian.arg("--pedantic");
}
if options.experimental {
lintian.arg("--display-experimental");
}
if options.show_overrides {
lintian.arg("--show-overrides");
}
if !options.suppress_tags.is_empty() {
lintian
.arg("--suppress-tags")
.arg(options.suppress_tags.join(","));
}
let output = match lintian.output() {
Ok(output) => output,
Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None),
Err(e) => return Err(format!("Could not run lintian: {e}")),
};
let stdout = String::from_utf8_lossy(&output.stdout);
let findings = parse(&stdout);
if !output.status.success() && findings.is_empty() {
return Err(format!(
"lintian exited with {} without reporting findings:\n{}",
output.status,
String::from_utf8_lossy(&output.stderr).trim()
));
}
Ok(Some(WrapperOutcome { findings, notes }))
}
/// Pack the tree into an ephemeral source package and return its `.dsc`.
///
/// Compression is deliberately weak (`-Zgzip -z1`): the artifact only ever
/// goes to lintian and is deleted with the temp directory, and xz on a
/// large tree dominates the whole run (measured: 9.8 s xz vs 2.7 s gzip on
/// a 111 MB tree). Locale-independent subprocess output: dpkg messages can
/// be localized, and the error sniffing relies on English wording.
fn pack(root: &Path, tmp: &TempDir) -> Result<PathBuf, String> {
// 3.0 (quilt) trees need the orig tarball(s) reachable from the working
// directory, and dpkg-source searches cwd — link them from the tree's
// parent, where pkh build / git ubuntu export-orig leave them.
link_orig_tarballs(root, tmp.path());
let output = Command::new("dpkg-source")
.env("LC_ALL", "C")
.args(["-b", "-Zgzip", "-z1"])
.arg(root)
.current_dir(tmp.path())
.output()
.map_err(|e| format!("Could not run dpkg-source: {e}"))?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
if stderr.to_lowercase().contains("uncommitted") {
return Err(
"The source tree has uncommitted changes that dpkg-source refuses \
to pack. Commit them first, or use --native to lint with pkh's \
native checks only."
.to_string(),
);
}
return Err(format!(
"dpkg-source -b failed, the tree may not be a valid source package:\n{}",
stderr.trim()
));
}
find_dsc(tmp.path())
}
/// The `pkh build` output matching the tree's current changelog entry, when
/// it exists and no tree content is newer than it: linting a stale artifact
/// would report the packaging of the past, so staleness forces a fresh pack.
fn usable_build_output(root: &Path, force_repack: bool) -> Option<PathBuf> {
if force_repack {
return None;
}
let dsc = expected_build_output(root)?;
let built = std::fs::metadata(&dsc).ok()?.modified().ok()?;
if tree_newer_than(root, built) {
return None;
}
Some(dsc)
}
/// The `pkh build` output path matching the tree's current changelog entry
/// (`../<source>_<version>.dsc`, pkh build's own naming), if it exists.
fn expected_build_output(root: &Path) -> Option<PathBuf> {
let entry =
crate::debian::changelog::parse_changelog_entry(&root.join("debian/changelog")).ok()?;
let dsc = root
.parent()?
.join(format!("{}_{}.dsc", entry.source, entry.version.no_epoch()));
std::fs::metadata(&dsc).ok()?;
Some(dsc)
}
/// Whether any tree content is newer than `built`. Skips `.git` and `.pc`:
/// commits and quilt bookkeeping churn their mtimes without touching what
/// the source package contains.
fn tree_newer_than(root: &Path, built: SystemTime) -> bool {
const SKIP: &[&str] = &[".git", ".pc"];
let mut stack = vec![root.to_path_buf()];
while let Some(dir) = stack.pop() {
let Ok(entries) = std::fs::read_dir(&dir) else {
continue;
};
for entry in entries.flatten() {
let name = entry.file_name();
if SKIP.iter().any(|skip| name.to_string_lossy() == *skip) {
continue;
}
match entry.file_type() {
Ok(ft) if ft.is_dir() => stack.push(entry.path()),
_ => {
let newer = entry
.metadata()
.ok()
.and_then(|m| m.modified().ok())
.is_some_and(|modified| modified > built);
if newer {
return true;
}
}
}
}
}
false
}
/// Symlink the orig (and orig component) tarballs for the tree's upstream
/// version from the tree's parent into `dest`, ignoring absence — native
/// trees have none, and missing tarballs surface as a dpkg-source error.
fn link_orig_tarballs(root: &Path, dest: &Path) {
let Some(parent) = root.parent() else {
return;
};
let Ok(entry) = crate::debian::changelog::parse_changelog_entry(&root.join("debian/changelog"))
else {
return;
};
let prefixes = [
format!("{}_{}.orig.tar.", entry.source, entry.version.upstream),
format!("{}_{}.orig-", entry.source, entry.version.upstream),
];
let Ok(entries) = std::fs::read_dir(parent) else {
return;
};
for candidate in entries.flatten() {
let name = candidate.file_name();
let name = name.to_string_lossy();
if !candidate.file_type().is_ok_and(|ft| ft.is_file())
|| !prefixes
.iter()
.any(|prefix| name.starts_with(prefix.as_str()))
{
continue;
}
let _ = std::os::unix::fs::symlink(parent.join(name.as_ref()), dest.join(name.as_ref()));
}
}
/// Parse lintian's output into findings. Tag lines carry the finding; `N:`
/// note lines following a tag line are its explanation (`--info` output) and
/// attach to it. Anything else is ignored.
fn parse(text: &str) -> Vec<Finding> {
let tag_line = Regex::new(r"^(?P<letter>[EWIPXOC]): (?P<rest>.*)$").expect("static regex");
let subject = Regex::new(
r"^(?P<pkg>\S+?)(?: (?P<ptype>source|binary|udeb|changes|buildinfo))?: (?P<tag>\S+)(?: (?P<details>.*))?$",
)
.expect("static regex");
let note_line = Regex::new(r"^N:(?: (?P<text>.*))?$").expect("static regex");
let mut findings: Vec<Finding> = Vec::new();
for line in text.lines() {
if let Some(note) = note_line.captures(line) {
// Attach to the finding above, like lintian lays out --info.
if let (Some(text), Some(last)) = (note.name("text"), findings.last_mut())
&& !text.as_str().trim().is_empty()
{
last.explanation.push(text.as_str().trim().to_string());
}
continue;
}
let Some(head) = tag_line.captures(line) else {
continue;
};
let Some(subject) = subject.captures(&head["rest"]) else {
continue;
};
findings.push(Finding {
letter: head["letter"].chars().next().unwrap_or('E'),
tag_name: subject["tag"].to_string(),
message: subject
.name("details")
.map_or(String::new(), |d| d.as_str().to_string()),
package: subject["pkg"].to_string(),
processable_type: subject.name("ptype").map(|p| p.as_str().to_string()),
explanation: Vec::new(),
origin: Origin::Lintian,
});
}
findings
}
/// The single `.dsc` the ephemeral source package produced.
fn find_dsc(dir: &Path) -> Result<PathBuf, String> {
let mut entries: Vec<PathBuf> = std::fs::read_dir(dir)
.map_err(|e| format!("Cannot read the temporary build directory: {e}"))?
.flatten()
.map(|entry| entry.path())
.filter(|path| path.extension().is_some_and(|ext| ext == "dsc"))
.collect();
match entries.pop() {
Some(dsc) => Ok(dsc),
None => Err(
"dpkg-source produced no .dsc; the tree may not be a valid source package".to_string(),
),
}
}
/// Temporary directory removed on drop; the hand-rolled stand-in for
/// `tempfile`, which is dev-only in this crate.
struct TempDir(PathBuf);
impl TempDir {
fn new() -> Result<TempDir, String> {
let unique = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0);
let path = std::env::temp_dir().join(format!("pkh-lint-{}-{unique}", std::process::id()));
std::fs::create_dir(&path)
.map_err(|e| format!("Could not create a temporary directory: {e}"))?;
Ok(TempDir(path))
}
fn path(&self) -> &Path {
&self.0
}
}
impl Drop for TempDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Captured from lintian 2.129.0ubuntu2.1 on a broken native source
/// package (default display level).
const SOURCE_OUTPUT: &str = "\
E: hello source: malformed-debian-changelog-version 0.1-1 (for native) [debian/changelog:1]
E: hello source: package-uses-debhelper-but-lacks-build-depends [debian/rules]
W: hello source: debhelper-but-no-misc-depends hello
W: hello source: debhelper-compat-file-is-missing
W: hello source: no-debian-copyright-in-source
";
/// Captured from lintian 2.129.0ubuntu2.1 on a binary package: binary
/// findings carry no processable type after the package name.
const BINARY_OUTPUT: &str = "\
E: badpkg: description-too-short test
E: badpkg: extended-description-is-empty
W: badpkg: empty-binary-package
W: badpkg: recommended-field badpkg_1.0-1_all.deb Priority
";
/// Captured with --show-overrides: overridden findings print as `O:`.
const OVERRIDDEN_OUTPUT: &str = "\
E: hello source: malformed-debian-changelog-version 0.1-1 (for native) [debian/changelog:1]
O: hello source: debhelper-compat-file-is-missing
";
/// Captured with --info: each tag line is followed by `N:` explanation
/// lines that belong to it.
const INFO_OUTPUT: &str = "\
N:
E: hello source: malformed-debian-changelog-version 0.1-1 (for native) [debian/changelog:1]
N:
N: The version string in the latest changelog entry was not parsed correctly.
N: Usually, that means it does not conform to policy.
N:
N:
E: hello source: package-uses-debhelper-but-lacks-build-depends [debian/rules]
N:
N: If a package uses debhelper, it must declare a Build-Depends on debhelper
N: or on the debhelper-compat virtual package. For example:
N:
";
#[test]
fn parses_source_output_with_type() {
let findings = parse(SOURCE_OUTPUT);
assert_eq!(findings.len(), 5);
let first = &findings[0];
assert_eq!(first.letter, 'E');
assert_eq!(first.package, "hello");
assert_eq!(first.processable_type.as_deref(), Some("source"));
assert_eq!(first.tag_name, "malformed-debian-changelog-version");
assert_eq!(first.message, "0.1-1 (for native) [debian/changelog:1]");
assert_eq!(findings[2].letter, 'W');
}
#[test]
fn parses_binary_output_without_type() {
let findings = parse(BINARY_OUTPUT);
assert_eq!(findings.len(), 4);
let first = &findings[0];
assert_eq!(first.package, "badpkg");
assert_eq!(first.processable_type, None);
assert_eq!(first.tag_name, "description-too-short");
// Details containing a file name with dots survive intact.
assert_eq!(findings[3].message, "badpkg_1.0-1_all.deb Priority");
}
#[test]
fn parses_overridden_lines() {
let findings = parse(OVERRIDDEN_OUTPUT);
assert_eq!(findings.len(), 2);
assert_eq!(findings[1].letter, 'O');
assert_eq!(findings[1].tag_name, "debhelper-compat-file-is-missing");
assert_eq!(findings[1].message, "");
}
#[test]
fn attaches_info_notes_to_the_preceding_finding() {
let findings = parse(INFO_OUTPUT);
assert_eq!(findings.len(), 2);
assert_eq!(
findings[0].explanation,
vec![
"The version string in the latest changelog entry was not parsed correctly.",
"Usually, that means it does not conform to policy.",
]
);
assert_eq!(findings[1].explanation.len(), 2);
assert!(findings[1].explanation[0].starts_with("If a package uses debhelper"));
}
#[test]
fn ignores_stray_lines() {
let findings =
parse("N: lintian ran\ngarbage line\nC: hello source: some-classification\n");
// The C: classification line parses (kept for JSON), garbage drops.
assert_eq!(findings.len(), 1);
assert_eq!(findings[0].letter, 'C');
}
/// A lintable tree at `<outer>/<name>` with a changelog entry
/// `pkg (1.0-1) resolute`; returns the outer dir (the tree's parent,
/// where pkh build would place artifacts).
fn tree() -> (tempfile::TempDir, std::path::PathBuf) {
let outer = tempfile::tempdir().unwrap();
let root = outer.path().join("pkg-1.0");
std::fs::create_dir_all(root.join("debian")).unwrap();
std::fs::write(
root.join("debian/changelog"),
"pkg (1.0-1) resolute; urgency=medium\n\n * x\n\n -- J <j@e.org> Sat, 19 Sep 2026 12:00:00 +0000\n",
)
.unwrap();
(outer, root)
}
fn dsc_of(outer: &tempfile::TempDir) -> std::path::PathBuf {
outer.path().join("pkg_1.0-1.dsc")
}
#[test]
fn current_build_output_is_reused() {
let (outer, root) = tree();
std::fs::write(dsc_of(&outer), "dummy dsc").unwrap();
// The dsc was written after every tree file: current.
assert_eq!(usable_build_output(&root, false), Some(dsc_of(&outer)));
// Forcing repack skips it.
assert_eq!(usable_build_output(&root, true), None);
}
#[test]
fn stale_or_mismatched_build_output_is_rejected() {
let (outer, root) = tree();
// A tree file written after the dsc makes the artifact stale. The
// sleep crosses the coarse clock tick mtimes are stamped with, so
// the control file is strictly newer than the dsc.
std::fs::write(dsc_of(&outer), "dummy dsc").unwrap();
std::thread::sleep(std::time::Duration::from_millis(10));
std::fs::write(root.join("debian/control"), "Source: pkg\n").unwrap();
assert_eq!(usable_build_output(&root, false), None);
// A dsc of a different version does not represent this tree.
let old_dsc = outer.path().join("pkg_0.9-1.dsc");
std::fs::write(&old_dsc, "dummy dsc").unwrap();
assert_eq!(usable_build_output(&root, false), None);
}
#[test]
fn missing_build_output_is_rejected() {
let (outer, root) = tree();
assert_eq!(usable_build_output(&root, false), None);
assert!(!dsc_of(&outer).exists());
}
#[test]
fn orig_tarballs_are_linked_for_packing() {
let (outer, root) = tree();
std::fs::write(outer.path().join("pkg_1.0.orig.tar.xz"), "orig").unwrap();
std::fs::write(outer.path().join("pkg_1.0.orig-data.tar.gz"), "comp").unwrap();
std::fs::write(outer.path().join("unrelated_1.0.orig.tar.xz"), "no").unwrap();
std::fs::write(outer.path().join("pkg_1.0-1.dsc"), "no").unwrap();
let dest = tempfile::tempdir().unwrap();
link_orig_tarballs(&root, dest.path());
assert!(
dest.path()
.join("pkg_1.0.orig.tar.xz")
.symlink_metadata()
.is_ok()
);
assert!(
dest.path()
.join("pkg_1.0.orig-data.tar.gz")
.symlink_metadata()
.is_ok()
);
assert!(!dest.path().join("unrelated_1.0.orig.tar.xz").exists());
// The dsc is not an orig tarball and must not be linked.
assert!(!dest.path().join("pkg_1.0-1.dsc").exists());
}
}
View File
+749 -195
View File
File diff suppressed because it is too large Load Diff
+946
View File
@@ -0,0 +1,946 @@
//! Generators for the common `debian/` files of a scaffolded package, plus
//! the orig tarball creation.
//!
//! Everything here renders in memory as [`OutputFile`]s; the caller writes
//! them all-or-nothing after checking for collisions (see
//! [`super::scaffold`]).
use std::collections::HashSet;
use std::path::Path;
use chrono::Datelike;
use tar::Builder;
use xz2::write::XzEncoder;
use super::options::{NewOptions, SourceFormat};
use super::templates::{OutputFile, Template};
/// `3.0 (quilt)` source format, the default when packaging an existing
/// project.
pub const SOURCE_FORMAT_QUILT: &str = "3.0 (quilt)";
/// `3.0 (native)` source format, the default for a fresh skeleton.
pub const SOURCE_FORMAT_NATIVE: &str = "3.0 (native)";
/// The three source formats pkh knows how to build.
pub const KNOWN_SOURCE_FORMATS: [&str; 3] = [SOURCE_FORMAT_QUILT, SOURCE_FORMAT_NATIVE, "1.0"];
/// Current Debian Policy version, written as the `Standards-Version` of the
/// generated `debian/control` (mandatory in the source stanza per policy).
/// Bump as policy evolves.
pub const DEBIAN_POLICY_VERSION: &str = "4.7.4";
/// Directory and file names excluded from the orig tarball, at any depth of
/// the tree.
const ORIG_EXCLUDE: &[&str] = &[
".git",
"debian",
"target",
"node_modules",
"__pycache__",
".venv",
];
/// Path of the orig tarball for `name`/`upstream_version` next to `tree`.
pub fn orig_tarball_path(
tree: &Path,
name: &str,
upstream_version: &str,
) -> Option<std::path::PathBuf> {
tree.parent()
.map(|parent| parent.join(format!("{name}_{upstream_version}.orig.tar.xz")))
}
/// Render every common `debian/` file of the package.
pub fn files(opts: &NewOptions, template: &Template) -> Vec<OutputFile> {
let mut files = vec![
source_format(opts),
changelog(opts),
control(opts, template),
rules(opts, template),
copyright(opts),
debian_gitignore(opts),
];
if opts.source_format == SourceFormat::Quilt {
files.push(local_options());
}
if opts.autopkgtest {
files.push(autopkgtest_control());
files.push(autopkgtest_smoke(opts));
}
if let Some(watch) = &opts.watch {
files.push(OutputFile::new("debian/watch", watch.clone()));
}
files
}
/// `debian/tests/control`: the autopkgtest smoke test definition.
fn autopkgtest_control() -> OutputFile {
OutputFile::new(
"debian/tests/control",
"Tests: smoke\nDepends: @\nRestrictions: allow-stderr\n",
)
}
/// `debian/tests/smoke`: run the installed command once; `--help` first,
/// `--version` as the fallback (some tools only answer one of them).
fn autopkgtest_smoke(opts: &NewOptions) -> OutputFile {
OutputFile::executable(
"debian/tests/smoke",
format!(
"#!/bin/sh\n\
set -e\n\
{command} --help >/dev/null 2>&1 || {command} --version\n",
command = opts.command,
),
)
}
/// `debian/source/format`: `3.0 (native)` for a skeleton by default,
/// `3.0 (quilt)` for an existing project; either can be forced with
/// `--native` / `--quilt`.
fn source_format(opts: &NewOptions) -> OutputFile {
OutputFile::new(
"debian/source/format",
format!("{}\n", opts.source_format.deb_string()),
)
}
/// `debian/source/local-options` with `single-debian-patch`, so later
/// upstream-tree edits stay representable as one `debian/patches/debian-changes-*`
/// patch instead of failing the build (quilt only). Common build-output
/// directories are excluded from the delta as well: compiling locally before
/// a source build must not turn `target/`, `node_modules/` or `.venv/`
/// binaries into unrepresentable changes (dpkg ignores `__pycache__` and
/// friends by default, but not those).
fn local_options() -> OutputFile {
OutputFile::new(
"debian/source/local-options",
"single-debian-patch\n\
extend-diff-ignore = ^target/\n\
extend-diff-ignore = ^node_modules/\n\
extend-diff-ignore = ^\\.venv/\n",
)
}
/// `debian/changelog`: the single initial entry, distribution UNRELEASED by
/// default (the dh_make convention: a fresh package is by definition not
/// ready for upload, and pkh skips signing for UNRELEASED), or the target
/// series with `--release`.
fn changelog(opts: &NewOptions) -> OutputFile {
let distribution = if opts.release {
opts.series.as_str()
} else {
crate::distro_info::UNRELEASED
};
let date = chrono::Local::now().format("%a, %d %b %Y %H:%M:%S %z");
OutputFile::new(
"debian/changelog",
format!(
"{name} ({version}) {distribution}; urgency=medium\n\
\n\
\x20 * Initial release.\n\
\n\
\x20-- {maintainer_name} <{maintainer_email}> {date}\n",
name = opts.name,
version = opts.full_version(),
distribution = distribution,
maintainer_name = opts.maintainer.0,
maintainer_email = opts.maintainer.1,
date = date,
),
)
}
/// Render a field whose values continue one per line (RFC822 continuation,
/// one leading space, commas between values, first value on the field line):
///
/// ```text
/// Build-Depends: debhelper-compat (= 13),
/// python3-all
/// ```
fn render_field(name: &str, values: &[String]) -> String {
let last = values.len() - 1;
let mut out = format!("{}: {}", name, values[0]);
if last > 0 {
out.push(',');
}
out.push('\n');
for (i, value) in values.iter().enumerate().skip(1) {
out.push_str(&format!(" {value}"));
if i != last {
out.push(',');
}
out.push('\n');
}
out
}
/// Render a free-text field body (long description, license paragraphs):
/// every line as a continuation, blank lines as ` .` (the deb822 encoding).
fn render_continuation_text(text: &str) -> String {
let mut out = String::new();
for line in text.lines() {
if line.trim().is_empty() {
out.push_str(" .\n");
} else {
out.push_str(&format!(" {line}\n"));
}
}
out
}
/// `debian/control`: one source stanza plus one binary stanza.
///
/// The binary package name is the source package name, the architecture
/// comes from the template (`all` for shell/empty), and a non-empty
/// `opts.depends` (the empty/metapackage flavor) lands in the binary
/// stanza's `Depends` field.
fn control(opts: &NewOptions, template: &Template) -> OutputFile {
let mut control = String::new();
// Source stanza.
control.push_str(&format!("Source: {}\n", opts.name));
control.push_str("Section: utils\n");
control.push_str("Priority: optional\n");
control.push_str(&format!(
"Maintainer: {} <{}>\n",
opts.maintainer.0, opts.maintainer.1
));
control.push_str("Rules-Requires-Root: no\n");
control.push_str(&format!("Standards-Version: {DEBIAN_POLICY_VERSION}\n"));
let mut build_depends = vec!["debhelper-compat (= 13)".to_string()];
build_depends.extend(template.build_depends(opts));
control.push_str(&render_field("Build-Depends", &build_depends));
for (key, value) in template.source_fields(opts) {
control.push_str(&format!("{key}: {value}\n"));
}
if let Some(homepage) = &opts.homepage {
control.push_str(&format!("Homepage: {homepage}\n"));
}
control.push('\n');
// Binary stanza.
control.push_str(&format!("Package: {}\n", opts.name));
control.push_str(&format!("Architecture: {}\n", template.architecture(opts)));
if !opts.depends.is_empty() {
control.push_str(&render_field("Depends", &opts.depends));
}
control.push_str(&format!("Description: {}\n", opts.summary));
control.push_str(&render_continuation_text(&opts.long_description));
OutputFile::new("debian/control", control)
}
/// `debian/rules`: the shebang and `%:` target whose recipe is the
/// template's dh line (plus the template's extra overrides, when any),
/// written with the executable bit.
fn rules(opts: &NewOptions, template: &Template) -> OutputFile {
let mut contents = format!("#!/usr/bin/make -f\n%:\n\t{}\n", template.rules_dh_line());
let extra = template.rules_extra(opts);
if !extra.is_empty() {
contents.push('\n');
contents.push_str(&extra);
if !contents.ends_with('\n') {
contents.push('\n');
}
}
OutputFile::executable("debian/rules", contents)
}
/// Short license-reference paragraph embedded in `debian/copyright`.
fn license_reference_paragraph(license: &super::options::License) -> String {
use super::options::License;
match license {
License::Custom(s) if s.eq_ignore_ascii_case("unknown") => {
"The licensing terms of this package are not known yet. \
Replace this paragraph with a proper license reference."
.to_string()
}
License::Custom(s) => format!(
"The package is distributed under the terms of the '{s}' license. \
Replace this paragraph with the full license reference."
),
known => format!(
"The package is distributed under the terms of the {} license. \
The full license text is available at <{}>.",
known.spdx(),
known.spdx_url()
),
}
}
/// `debian/copyright` in the DEP-5 machine-readable format: header, the
/// `Files: *` stanza covering the current year, and a standalone license
/// stanza with a short reference paragraph.
fn copyright(opts: &NewOptions) -> OutputFile {
let year = chrono::Local::now().year();
let mut out = String::new();
out.push_str("Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/\n");
out.push_str(&format!("Upstream-Name: {}\n", opts.name));
if let Some(homepage) = &opts.homepage {
out.push_str(&format!("Source: {homepage}\n"));
}
out.push('\n');
out.push_str("Files: *\n");
out.push_str(&format!(
"Copyright: {} {} <{}>\n",
year, opts.maintainer.0, opts.maintainer.1
));
out.push_str(&format!("License: {}\n", opts.license.spdx()));
out.push_str(&render_continuation_text(&license_reference_paragraph(
&opts.license,
)));
out.push('\n');
out.push_str(&format!("License: {}\n", opts.license.spdx()));
out.push_str(&render_continuation_text(&license_reference_paragraph(
&opts.license,
)));
OutputFile::new("debian/copyright", out)
}
/// `debian/.gitignore`: the debhelper build artifacts. The patterns are
/// relative to `debian/` itself (a `debian/`-prefixed pattern would be
/// anchored to `debian/debian/` inside this file, per gitignore(5)).
fn debian_gitignore(opts: &NewOptions) -> OutputFile {
OutputFile::new(
"debian/.gitignore",
format!(
"files\n\
.debhelper/\n\
*.log\n\
{}/\n\
debhelper-build-stamp\n\
*.substvars\n",
opts.name
),
)
}
/// Entries of the root `.gitignore` written in skeleton mode (build
/// artifacts, next to the tree).
pub const ROOT_GITIGNORE_ENTRIES: [&str; 6] = [
"*.deb",
"*.dsc",
"*.changes",
"*.buildinfo",
"*.tar.xz",
"target/",
];
/// Comment heading a root `.gitignore` freshly created by pkh (the skeleton
/// build-artifact section).
pub const ROOT_GITIGNORE_HEADER: &str = "# pkh build artifacts";
/// Merge `entries` into the root `.gitignore` contents `existing` (the
/// current file contents, when there is one): missing entries are appended,
/// an existing file is never overwritten just to duplicate entries. A fresh
/// file is headed by the `header` comment when one is given; appending to a
/// user file adds bare entries. Returns the new contents, or `None` when
/// nothing has to be written.
pub fn merge_gitignore_entries(
existing: Option<&str>,
entries: &[&str],
header: Option<&str>,
) -> Option<String> {
let have: HashSet<&str> = existing
.map(|content| content.lines().map(str::trim).collect())
.unwrap_or_default();
let missing: Vec<&str> = entries
.iter()
.copied()
.filter(|entry| !have.contains(entry))
.collect();
if missing.is_empty() {
return None;
}
let mut out = existing.unwrap_or("").to_string();
if !out.is_empty() && !out.ends_with('\n') {
out.push('\n');
}
// Section comment only for a fresh file; appending to a user file adds
// bare entries.
if existing.is_none()
&& let Some(header) = header
{
out.push_str(header);
out.push('\n');
}
for entry in missing {
out.push_str(entry);
out.push('\n');
}
Some(out)
}
/// Create `../<name>_<upstream_version>.orig.tar.xz` containing the tree,
/// excluding `debian/` and VCS/build directories, so the first
/// `dpkg-source -b` (quilt) succeeds immediately. Refuses to overwrite an
/// existing tarball.
pub fn create_orig_tarball(
tree: &Path,
name: &str,
upstream_version: &str,
) -> Result<std::path::PathBuf, Box<dyn std::error::Error>> {
create_orig_tarball_excluding(tree, name, upstream_version, false)
}
/// [`create_orig_tarball`] with the generated `vendor/` directory of a
/// vendored rust package excluded from the snapshot: its contents travel in
/// the separate `<name>_<uver>.orig-vendor.tar.xz` component instead (see
/// [`super::orig`]), so they can be regenerated independently of the
/// upstream sources.
pub fn create_orig_tarball_excluding(
tree: &Path,
name: &str,
upstream_version: &str,
exclude_vendor: bool,
) -> Result<std::path::PathBuf, Box<dyn std::error::Error>> {
let tarball_path = orig_tarball_path(tree, name, upstream_version).ok_or_else(|| {
format!(
"cannot determine the parent directory of '{}'",
tree.display()
)
})?;
if tarball_path.exists() {
return Err(format!(
"'{}' already exists: pkh new refuses to overwrite it. \
Remove it first, or pass --native to skip the orig tarball.",
tarball_path.display()
)
.into());
}
let file = std::fs::File::create(&tarball_path)?;
let encoder = XzEncoder::new(file, 6);
let mut builder = Builder::new(encoder);
// Deterministic-ish ordering: sort entries by name at every level.
let prefix = format!("{name}-{upstream_version}");
// The single top-level directory dpkg-source expects.
builder.append_dir(&prefix, tree)?;
let top_excludes: &[&str] = if exclude_vendor { &["vendor"] } else { &[] };
append_tree(&mut builder, tree, &prefix, 0, ORIG_EXCLUDE, top_excludes)?;
builder
.finish()
.map_err(|e| format!("failed to write '{}': {}", tarball_path.display(), e))?;
log::info!(
"Created orig tarball {}",
crate::report::display_path(&tarball_path)
);
Ok(tarball_path)
}
/// Recursively append `dir` to the archive under `archive_path`, skipping
/// non-regular files, the names of `excludes` at any depth, the `debian/`
/// directory and the names of `top_excludes` at the top level (depth 0).
pub(crate) fn append_tree(
builder: &mut Builder<XzEncoder<std::fs::File>>,
dir: &Path,
archive_path: &str,
depth: usize,
excludes: &[&str],
top_excludes: &[&str],
) -> Result<(), Box<dyn std::error::Error>> {
let mut entries: Vec<std::fs::DirEntry> = std::fs::read_dir(dir)?.collect::<Result<_, _>>()?;
entries.sort_by_key(|entry| entry.file_name());
for entry in entries {
let path = entry.path();
let file_name = entry.file_name();
let name = file_name.to_string_lossy().into_owned();
if depth == 0 && (name == "debian" || top_excludes.contains(&name.as_str())) {
continue;
}
if excludes.contains(&name.as_str()) {
continue;
}
let entry_archive_path = format!("{archive_path}/{name}");
let metadata = std::fs::metadata(&path)
.map_err(|e| format!("cannot stat '{}': {}", path.display(), e))?;
if metadata.is_dir() {
builder.append_dir(&entry_archive_path, &path)?;
append_tree(
builder,
&path,
&entry_archive_path,
depth + 1,
excludes,
top_excludes,
)?;
} else if metadata.is_file() {
// The mode (including the exec bit) travels through the header.
let mut header = tar::Header::new_gnu();
header.set_metadata(&metadata);
header.set_size(metadata.len());
let file = std::fs::File::open(&path)
.map_err(|e| format!("cannot read '{}': {}", path.display(), e))?;
builder
.append_data(&mut header, &entry_archive_path, file)
.map_err(|e| format!("cannot add '{}' to the tarball: {}", path.display(), e))?;
} else {
// Sockets, fifos, devices have no business in an orig tarball.
log::warn!(
"Skipping non-regular file '{}' while creating the orig tarball",
path.display()
);
}
}
Ok(())
}
/// Write an in-memory file list to `tree`, creating parent directories and
/// applying the executable bit. Callers must have checked collisions first.
pub(crate) fn write_files(
tree: &Path,
files: &[OutputFile],
) -> Result<(), Box<dyn std::error::Error>> {
use std::os::unix::fs::PermissionsExt;
for file in files {
let path = tree.join(&file.path);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&path, &file.contents)?;
if file.executable {
let mut permissions = std::fs::metadata(&path)?.permissions();
permissions.set_mode(0o755);
std::fs::set_permissions(&path, permissions)?;
}
log::debug!("Wrote {}", path.display());
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::new::options::{License, SourceDir, TemplateId};
fn opts() -> NewOptions {
NewOptions {
name: "mytool".into(),
template: TemplateId::SHELL,
source_dir: SourceDir::Skeleton,
upstream_version: "0.1.0".into(),
revision: 1,
summary: "A tool that does one thing well".into(),
long_description: "A tool that does one thing well".into(),
homepage: Some("https://example.com/mytool".into()),
license: License::Mit,
command: "mytool".into(),
maintainer: ("Jane Doe".into(), "jane@example.com".into()),
dist: "ubuntu".into(),
series: "resolute".into(),
release: false,
depends: Vec::new(),
source_format: SourceFormat::Quilt,
orig: None,
git: true,
autopkgtest: false,
pkg_config: false,
watch: None,
}
}
#[test]
fn source_format_and_local_options() {
let o = opts();
let files = super::files(&o, crate::new::templates::get(TemplateId::SHELL).unwrap());
let find = |path: &str| {
files
.iter()
.find(|f| f.path == path)
.unwrap_or_else(|| panic!("{path} missing"))
};
assert_eq!(find("debian/source/format").contents, "3.0 (quilt)\n");
assert_eq!(
find("debian/source/local-options").contents,
"single-debian-patch\n\
extend-diff-ignore = ^target/\n\
extend-diff-ignore = ^node_modules/\n\
extend-diff-ignore = ^\\.venv/\n"
);
let native = NewOptions {
source_format: SourceFormat::Native,
..opts()
};
let files = super::files(
&native,
crate::new::templates::get(TemplateId::SHELL).unwrap(),
);
assert!(
files
.iter()
.all(|f| f.path != "debian/source/local-options")
);
assert_eq!(
files
.iter()
.find(|f| f.path == "debian/source/format")
.unwrap()
.contents,
"3.0 (native)\n"
);
}
#[test]
fn changelog_rendering_and_parse() {
let o = opts();
let changelog = super::changelog(&o);
assert_eq!(changelog.path, "debian/changelog");
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("changelog");
std::fs::write(&path, &changelog.contents).unwrap();
let (source, version, distribution) =
crate::changelog::parse_changelog_header(&path).unwrap();
assert_eq!(source, "mytool");
assert_eq!(version, "0.1.0-1");
assert_eq!(distribution, "UNRELEASED");
// dpkg-style zero-padded RFC2822 date in the trailer.
assert!(
changelog
.contents
.contains(" -- Jane Doe <jane@example.com> ")
);
let date_line = changelog
.contents
.lines()
.find(|l| l.starts_with(" -- "))
.unwrap();
let date = date_line.rsplit_once(" ").unwrap().1;
// `%d` is zero-padded: positions 5-6 must be the two-digit day
// (e.g. "Tue, 05 Sep 2026 ...").
assert!(date[5..7].bytes().all(|b| b.is_ascii_digit()));
// --release writes the target series.
let released = NewOptions {
release: true,
..opts()
};
std::fs::write(&path, super::changelog(&released).contents).unwrap();
let (_, _, distribution) = crate::changelog::parse_changelog_header(&path).unwrap();
assert_eq!(distribution, "resolute");
}
#[test]
fn control_rendering_and_parse() {
let o = opts();
let control = super::control(&o, crate::new::templates::get(TemplateId::SHELL).unwrap());
// RFC822 continuation: first dep on the field line, the rest indented.
assert!(
control
.contents
.contains("Build-Depends: debhelper-compat (= 13)\n")
);
let parsed = crate::debian::ControlInfo::parse_content(&control.contents).unwrap();
assert_eq!(parsed.source_name(), "mytool");
assert_eq!(parsed.source.get("Section"), Some("utils"));
assert_eq!(parsed.source.get("Priority"), Some("optional"));
assert_eq!(parsed.source.get("Rules-Requires-Root"), Some("no"));
assert_eq!(
parsed.source.get("Standards-Version"),
Some(DEBIAN_POLICY_VERSION)
);
assert_eq!(
parsed.source.get("Homepage"),
Some("https://example.com/mytool")
);
assert_eq!(parsed.binaries.len(), 1);
assert_eq!(parsed.binaries[0].get("Package"), Some("mytool"));
assert_eq!(parsed.binaries[0].get("Architecture"), Some("all"));
assert_eq!(
parsed.binaries[0].get("Description"),
Some("A tool that does one thing well\nA tool that does one thing well")
);
// Without homepage both the control Homepage field and the DEP-5
// Source field are absent (the stanza's leading `Source:` line is
// still there of course).
let o = NewOptions {
homepage: None,
..opts()
};
let control = super::control(&o, crate::new::templates::get(TemplateId::SHELL).unwrap());
assert!(!control.contents.contains("Homepage:"));
let parsed = crate::debian::ControlInfo::parse_content(&control.contents).unwrap();
assert!(parsed.source.get("Homepage").is_none());
}
#[test]
fn rules_is_executable_minimal_makefile() {
let o = opts();
let rules = super::rules(&o, crate::new::templates::get(TemplateId::SHELL).unwrap());
assert!(rules.executable);
assert_eq!(rules.contents, "#!/usr/bin/make -f\n%:\n\tdh $@\n");
}
#[test]
fn extra_files_autopkgtest_and_watch() {
let mut o = opts();
o.autopkgtest = true;
o.watch = Some(
"version=4\nhttps://github.com/example/mytool/releases .*/v?@ANY_VERSION@\\.tar\\.gz\n"
.to_string(),
);
let files = super::files(&o, crate::new::templates::get(TemplateId::SHELL).unwrap());
let find = |path: &str| {
files
.iter()
.find(|f| f.path == path)
.unwrap_or_else(|| panic!("{path} missing"))
};
let control = find("debian/tests/control");
assert_eq!(
control.contents,
"Tests: smoke\nDepends: @\nRestrictions: allow-stderr\n"
);
let smoke = find("debian/tests/smoke");
assert!(smoke.executable);
assert!(smoke.contents.starts_with("#!/bin/sh\nset -e\n"));
assert!(
smoke
.contents
.contains("mytool --help >/dev/null 2>&1 || mytool --version")
);
assert_eq!(
find("debian/watch").contents,
"version=4\nhttps://github.com/example/mytool/releases .*/v?@ANY_VERSION@\\.tar\\.gz\n"
);
// Without the extras none of the files are rendered.
let plain = super::files(
&opts(),
crate::new::templates::get(TemplateId::SHELL).unwrap(),
);
assert!(!plain.iter().any(|f| f.path.starts_with("debian/tests")));
assert!(!plain.iter().any(|f| f.path == "debian/watch"));
}
#[test]
fn copyright_is_dep5() {
let c = super::copyright(&opts());
assert!(c.contents.starts_with(
"Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/\n"
));
assert!(c.contents.contains("Upstream-Name: mytool\n"));
assert!(c.contents.contains("Source: https://example.com/mytool\n"));
assert!(c.contents.contains("Files: *\n"));
assert!(c.contents.contains("License: MIT\n"));
assert!(c.contents.contains(&format!(
"Copyright: {} Jane Doe <jane@example.com>\n",
chrono::Local::now().year()
)));
assert!(c.contents.contains("https://spdx.org/licenses/MIT.html"));
// Unknown license: honest reference paragraph, still valid deb822.
let o = NewOptions {
license: License::Custom("unknown".into()),
..opts()
};
let c = super::copyright(&o);
assert!(c.contents.contains("not known yet"));
assert!(crate::debian::parse_paragraphs(&c.contents).len() >= 3);
}
#[test]
fn debian_gitignore_contents() {
let g = super::debian_gitignore(&opts());
assert_eq!(
g.contents,
"files\n.debhelper/\n*.log\nmytool/\n\
debhelper-build-stamp\n*.substvars\n"
);
}
#[test]
fn gitignore_merge() {
// Fresh file: header + all entries.
let fresh =
merge_gitignore_entries(None, &ROOT_GITIGNORE_ENTRIES, Some(ROOT_GITIGNORE_HEADER))
.unwrap();
assert!(fresh.starts_with("# pkh build artifacts\n"));
for entry in ROOT_GITIGNORE_ENTRIES {
assert!(fresh.contains(entry), "{entry} missing");
}
// A fresh file without a header carries the bare entries.
assert_eq!(
merge_gitignore_entries(None, &["a/", "b"], None).unwrap(),
"a/\nb\n"
);
// Existing file: only the missing entries are appended, nothing lost.
let existing = "*.deb\nnode_modules/\n";
let merged = merge_gitignore_entries(
Some(existing),
&ROOT_GITIGNORE_ENTRIES,
Some(ROOT_GITIGNORE_HEADER),
)
.unwrap();
assert!(merged.starts_with(existing));
assert!(merged.contains("*.dsc\n"));
assert!(!merged.contains("*.deb\n*.deb"));
// Everything already there: nothing to write.
let full: String = ROOT_GITIGNORE_ENTRIES
.iter()
.map(|e| format!("{e}\n"))
.collect();
assert!(
merge_gitignore_entries(
Some(&full),
&ROOT_GITIGNORE_ENTRIES,
Some(ROOT_GITIGNORE_HEADER),
)
.is_none()
);
}
/// The vendoring entries of the rust template merge into an existing
/// user `.gitignore` like any other entry set: appended after the
/// user's lines, no header comment, idempotent.
#[test]
fn gitignore_merge_appends_template_entries() {
let entries = ["vendor/", ".cargo/config.toml"];
let merged =
merge_gitignore_entries(Some("# my project\n*.log\n"), &entries, None).unwrap();
assert_eq!(merged, "# my project\n*.log\nvendor/\n.cargo/config.toml\n");
// Already ignored: nothing to write.
assert!(
merge_gitignore_entries(Some("vendor/\n.cargo/config.toml\n"), &entries, None)
.is_none()
);
}
#[test]
fn orig_tarball_layout() {
let dir = tempfile::tempdir().unwrap();
let tree = dir.path().join("mytool");
std::fs::create_dir_all(tree.join("debian")).unwrap();
std::fs::create_dir_all(tree.join("target")).unwrap();
std::fs::create_dir_all(tree.join("src/nested")).unwrap();
std::fs::write(tree.join("debian/control"), "control").unwrap();
std::fs::write(tree.join("target/artifact"), "junk").unwrap();
std::fs::write(tree.join("src/nested/code.txt"), "code").unwrap();
let tarball = create_orig_tarball(&tree, "mytool", "0.1.0").unwrap();
assert_eq!(tarball, dir.path().join("mytool_0.1.0.orig.tar.xz"));
assert!(tarball.exists());
let file = std::fs::File::open(&tarball).unwrap();
let mut archive = tar::Archive::new(xz2::read::XzDecoder::new(file));
let mut names: Vec<String> = archive
.entries()
.unwrap()
.map(|e| e.unwrap().path().unwrap().to_string_lossy().into_owned())
.collect();
// The tree prefix and the nested file are there...
assert!(
names
.iter()
.any(|n| n.trim_end_matches('/') == "mytool-0.1.0")
);
assert!(
names
.iter()
.any(|n| n == "mytool-0.1.0/src/nested/code.txt")
);
// ...but debian/, target/ and other excluded names are not.
assert!(!names.iter().any(|n| n.contains("debian")));
assert!(!names.iter().any(|n| n.contains("target")));
names.sort();
}
#[test]
fn orig_tarball_refuses_overwrite() {
let dir = tempfile::tempdir().unwrap();
let tree = dir.path().join("mytool");
std::fs::create_dir_all(&tree).unwrap();
std::fs::write(dir.path().join("mytool_0.1.0.orig.tar.xz"), b"existing").unwrap();
let err = create_orig_tarball(&tree, "mytool", "0.1.0").unwrap_err();
assert!(err.to_string().contains("already exists"));
}
/// The vendored-rust variant excludes the top-level `vendor/` (it
/// travels in the orig-vendor component) but keeps unrelated trees.
#[test]
fn orig_tarball_vendor_exclusion() {
let dir = tempfile::tempdir().unwrap();
let tree = dir.path().join("mytool");
std::fs::create_dir_all(tree.join("vendor/serde/src")).unwrap();
std::fs::create_dir_all(tree.join("src/vendor")).unwrap();
std::fs::write(tree.join("vendor/serde/src/lib.rs"), "code").unwrap();
std::fs::write(tree.join("src/vendor/mod.rs"), "code").unwrap();
std::fs::write(tree.join("Cargo.toml"), "[package]").unwrap();
let tarball = create_orig_tarball_excluding(&tree, "mytool", "0.1.0", true).unwrap();
let mut archive = tar::Archive::new(xz2::read::XzDecoder::new(
std::fs::File::open(&tarball).unwrap(),
));
let names: Vec<String> = archive
.entries()
.unwrap()
.map(|e| e.unwrap().path().unwrap().to_string_lossy().into_owned())
.collect();
// The generated vendored tree is out...
assert!(
!names.iter().any(|n| n.starts_with("mytool-0.1.0/vendor")),
"{names:?}"
);
// ...an unrelated nested vendor/ stays in...
assert!(
names.iter().any(|n| n == "mytool-0.1.0/src/vendor/mod.rs"),
"{names:?}"
);
// ...and normal files are unaffected.
assert!(
names.iter().any(|n| n == "mytool-0.1.0/Cargo.toml"),
"{names:?}"
);
}
#[test]
fn write_files_sets_exec_bit_and_parents() {
let dir = tempfile::tempdir().unwrap();
let files = vec![
OutputFile::new("a/b/c.txt", "deep"),
OutputFile::executable("debian/rules", "#!/usr/bin/make -f\n"),
];
write_files(dir.path(), &files).unwrap();
assert_eq!(
std::fs::read_to_string(dir.path().join("a/b/c.txt")).unwrap(),
"deep"
);
use std::os::unix::fs::PermissionsExt;
let mode = std::fs::metadata(dir.path().join("debian/rules"))
.unwrap()
.permissions()
.mode();
assert_eq!(mode & 0o777, 0o755);
}
}
+367
View File
@@ -0,0 +1,367 @@
//! Project detection for `pkh new`: which template matches an existing
//! source directory.
//!
//! The rule set is deliberately simple and table-driven (highest precedence
//! first):
//!
//! 1. the `detect.files` marker files declared by the template manifests
//! (`data/templates/<id>/manifest.yml`, in registry order: `Cargo.toml`,
//! `pyproject.toml`/`setup.py`/`setup.cfg`, `meson.build`,
//! `CMakeLists.txt`, `configure.ac`, `go.mod`, `Makefile`) looked for at
//! the top level of the directory — more than one distinct template
//! matching is [`Detection::Ambiguous`]; templates without markers
//! (shell: the single-script heuristic below; empty: never detected)
//! declare none,
//! 2. otherwise, exactly one top-level script (a `*.sh` file, or a file
//! whose first line is a `#!` shebang) → [`TemplateId::SHELL`],
//! several scripts or none → nothing,
//! 3. otherwise [`Detection::Empty`].
//!
//! Detection only looks at the top level on purpose: source files below
//! `src/` etc. carry no extra signal (a `src/main.rs` without `Cargo.toml`
//! is not a Rust project pkh can package), and recursion would turn stray
//! vendored files into false matches.
use std::path::Path;
use regex::Regex;
use super::licenses;
use super::options::TemplateId;
use super::templates;
/// Outcome of the detection.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Detection {
/// Exactly one template matches.
Single(TemplateId),
/// Several templates match; the caller must ask (wizard) or demand an
/// explicit `--lang`.
Ambiguous(Vec<TemplateId>),
/// Nothing recognized.
Empty,
}
/// Detect the template matching the project in `dir`: the manifests'
/// marker files in registry order (the detection priority), then the
/// shell single-script heuristic.
pub fn detect(dir: &Path) -> Detection {
let mut hits: Vec<TemplateId> = Vec::new();
for template in templates::all() {
let markers = template.detect_files();
if !markers.is_empty()
&& markers.iter().any(|marker| dir.join(marker).exists())
&& !hits.contains(&template.id())
{
hits.push(template.id());
}
}
match hits.as_slice() {
[] => {}
[only] => return Detection::Single(*only),
_ => return Detection::Ambiguous(hits),
}
if single_script(dir).is_some() {
Detection::Single(TemplateId::SHELL)
} else {
Detection::Empty
}
}
/// The single top-level script of `dir`, if there is exactly one: a file
/// with the `.sh` extension, or whose first line starts with `#!`. Returns
/// `None` when there are zero or several candidates.
pub fn single_script(dir: &Path) -> Option<std::path::PathBuf> {
let mut found: Option<std::path::PathBuf> = None;
let entries = std::fs::read_dir(dir).ok()?;
for entry in entries.flatten() {
let path = entry.path();
if !path.is_file() {
continue;
}
let name = entry.file_name();
let name = name.to_string_lossy();
// Hidden files and packaging leftovers carry no signal.
if name.starts_with('.') {
continue;
}
let is_script = name.ends_with(".sh") || has_shebang(&path);
if is_script {
if found.is_some() {
return None;
}
found = Some(path);
}
}
found
}
/// Whether the first line of the file starts with `#!`.
fn has_shebang(path: &Path) -> bool {
let Ok(content) = std::fs::read(path) else {
return false;
};
content.starts_with(b"#!")
}
/// Sniff the license of the project in `dir` from its `LICENSE`/`COPYING`
/// file: an `SPDX-License-Identifier:` line wins, otherwise the text is
/// matched against the marker sets of the bundled license table
/// (`data/licenses.yml`: MIT, BSD-2/3, Apache-2.0, GPL-2/3, LGPL-2.1/3,
/// ISC). `None` when no license file exists or nothing recognizable is
/// found.
pub fn sniff_license(dir: &Path) -> Option<String> {
let content = licenses::detect_files()
.find_map(|name| std::fs::read_to_string(dir.join(name)).ok())
// Case variants and suffixes (LICENSE-MIT, LICENCE, cpYING…): the
// first top-level file whose name looks like a license notice.
.or_else(|| {
let mut candidates: Vec<std::path::PathBuf> = std::fs::read_dir(dir)
.ok()?
.flatten()
.map(|entry| entry.path())
.filter(|path| {
path.is_file()
&& path
.file_name()
.and_then(|name| name.to_str())
.is_some_and(|name| {
let name = name.to_ascii_uppercase();
// American and British spellings both count.
name.starts_with("LICENSE")
|| name.starts_with("LICENCE")
|| name.starts_with("COPYING")
})
})
.collect();
candidates.sort();
std::fs::read_to_string(candidates.into_iter().next()?).ok()
})?;
// An explicit SPDX identifier is the most reliable signal.
static SPDX_REGEX: std::sync::OnceLock<Regex> = std::sync::OnceLock::new();
let spdx = SPDX_REGEX.get_or_init(|| {
Regex::new(r"(?i)SPDX-License-Identifier\s*:\s*([A-Za-z0-9+.\- ]+)").unwrap()
});
if let Some(id) = spdx
.captures(&content)
.and_then(|caps| caps.get(1))
.map(|id| id.as_str().trim_end().to_string())
.filter(|id| !id.is_empty())
{
return Some(id);
}
// The recognizable-license markers live in the bundled table; the
// LICENSE_TEXTS test below is their behavioral lock.
let text = content.to_ascii_lowercase();
licenses::detect_from_text(&text).map(str::to_string)
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::tempdir;
fn touch(dir: &Path, name: &str) {
std::fs::write(dir.join(name), "x").unwrap();
}
#[test]
fn marker_files_map_to_templates() {
let cases = [
("Cargo.toml", TemplateId::RUST),
("pyproject.toml", TemplateId::PYTHON),
("setup.py", TemplateId::PYTHON),
("setup.cfg", TemplateId::PYTHON),
("meson.build", TemplateId::MESON),
("CMakeLists.txt", TemplateId::CMAKE),
("configure.ac", TemplateId::AUTOTOOLS),
("go.mod", TemplateId::GO),
("Makefile", TemplateId::MAKEFILE),
];
for (marker, expected) in cases {
let dir = tempdir().unwrap();
touch(dir.path(), marker);
assert_eq!(detect(dir.path()), Detection::Single(expected), "{marker}");
}
}
#[test]
fn multiple_markers_are_ambiguous() {
let dir = tempdir().unwrap();
touch(dir.path(), "Cargo.toml");
touch(dir.path(), "Makefile");
assert_eq!(
detect(dir.path()),
Detection::Ambiguous(vec![TemplateId::RUST, TemplateId::MAKEFILE])
);
let dir = tempdir().unwrap();
touch(dir.path(), "pyproject.toml");
touch(dir.path(), "setup.py");
// Both markers map to the same template: one hit, not ambiguous.
assert_eq!(detect(dir.path()), Detection::Single(TemplateId::PYTHON));
let dir = tempdir().unwrap();
touch(dir.path(), "meson.build");
touch(dir.path(), "CMakeLists.txt");
assert_eq!(
detect(dir.path()),
Detection::Ambiguous(vec![TemplateId::MESON, TemplateId::CMAKE])
);
}
#[test]
fn single_script_is_shell() {
// .sh extension.
let dir = tempdir().unwrap();
touch(dir.path(), "run.sh");
assert_eq!(detect(dir.path()), Detection::Single(TemplateId::SHELL));
// Shebang without extension.
let dir = tempdir().unwrap();
std::fs::write(dir.path().join("run"), "#!/usr/bin/env python3\n").unwrap();
assert_eq!(detect(dir.path()), Detection::Single(TemplateId::SHELL));
// Two scripts: not exactly one, nothing recognized.
let dir = tempdir().unwrap();
touch(dir.path(), "a.sh");
touch(dir.path(), "b.sh");
assert_eq!(detect(dir.path()), Detection::Empty);
// Plain files without shebang are not scripts.
let dir = tempdir().unwrap();
touch(dir.path(), "README");
assert_eq!(detect(dir.path()), Detection::Empty);
}
#[test]
fn nothing_matches_is_empty() {
let dir = tempdir().unwrap();
assert_eq!(detect(dir.path()), Detection::Empty);
// Nonexistent directory: empty, not a panic.
let dir = tempdir().unwrap();
assert_eq!(detect(&dir.path().join("missing")), Detection::Empty);
}
#[test]
fn hidden_files_and_subdirs_are_ignored() {
let dir = tempdir().unwrap();
std::fs::create_dir(dir.path().join("subdir.sh")).unwrap();
std::fs::write(dir.path().join(".hidden.sh"), "#!/bin/sh\n").unwrap();
// The only "real" script candidate is in a subdir or hidden: no hit.
assert_eq!(detect(dir.path()), Detection::Empty);
}
/// Distinctive (shortened) excerpts of the recognizable license texts:
/// the behavioral lock of the marker sets in `data/licenses.yml` — a
/// bad marker edit fails here, not on real packages.
const LICENSE_TEXTS: [(&str, &str); 11] = [
(
"MIT",
"MIT License\n\nPermission is hereby granted, free of charge, to any person",
),
("Apache-2.0", "Apache License\nVersion 2.0, January 2004"),
(
"GPL-2.0+",
"GNU GENERAL PUBLIC LICENSE\nVersion 2, June 1991\neither version 2 of the License",
),
(
"GPL-3.0+",
"GNU GENERAL PUBLIC LICENSE\nVersion 3, 29 June 2007",
),
(
"LGPL-2.1+",
"GNU LESSER GENERAL PUBLIC LICENSE\nVersion 2.1, February 1999",
),
(
"LGPL-3.0+",
"GNU LESSER GENERAL PUBLIC LICENSE\nVersion 3, 29 June 2007",
),
(
"BSD-2-Clause",
"Redistribution and use in source and binary forms, with or without\nmodification, are permitted",
),
(
"BSD-3-Clause",
"Redistribution and use in source and binary forms, with or without\nmay be used to endorse or promote products",
),
(
"ISC",
"ISC License\nPermission to use, copy, modify, and/or distribute this software",
),
// Dual-licensed preamble: the GPL reference outranks the MIT
// boilerplate, like the old hardcoded cascade decided.
(
"GPL-2.0+",
"MIT License\n\nAlternatively, under the terms of the GNU General Public License,\
\nversion 2 of the License.",
),
// LGPL text naming both versions: the 2.1 wording wins.
(
"LGPL-2.1+",
"GNU LESSER GENERAL PUBLIC LICENSE\nVersion 2.1, February 1999\n\
This is version 2.1; version 3 is available separately.",
),
];
#[test]
fn sniff_license_recognizes_license_files() {
for (expected, text) in LICENSE_TEXTS {
// Every candidate file name is looked at.
for name in ["LICENSE", "COPYING", "LICENSE.md", "COPYING.txt"] {
let dir = tempdir().unwrap();
std::fs::write(dir.path().join(name), text).unwrap();
assert_eq!(
sniff_license(dir.path()).as_deref(),
Some(expected),
"{name}: {expected}"
);
}
}
}
#[test]
fn sniff_license_prefers_spdx_identifier() {
let dir = tempdir().unwrap();
std::fs::write(
dir.path().join("LICENSE"),
"Custom terms here\nSPDX-License-Identifier: Zlib\n",
)
.unwrap();
assert_eq!(sniff_license(dir.path()).as_deref(), Some("Zlib"));
}
#[test]
fn sniff_license_handles_case_variants_and_missing_files() {
// Unusual spelling found through the directory scan.
let dir = tempdir().unwrap();
std::fs::write(
dir.path().join("Licence.TXT"),
"Permission is hereby granted, free of charge",
)
.unwrap();
assert_eq!(sniff_license(dir.path()).as_deref(), Some("MIT"));
// Exact candidates win over the directory scan (LICENSE before
// LICENSE.blurb).
let dir = tempdir().unwrap();
std::fs::write(
dir.path().join("LICENSE.blurb"),
"GNU GENERAL PUBLIC LICENSE\nVersion 3, 29 June 2007",
)
.unwrap();
std::fs::write(dir.path().join("LICENSE"), "MIT License").unwrap();
assert_eq!(sniff_license(dir.path()).as_deref(), Some("MIT"));
// Unrecognizable or missing text: silent None.
let dir = tempdir().unwrap();
std::fs::write(dir.path().join("LICENSE"), "do whatever you want\n").unwrap();
assert_eq!(sniff_license(dir.path()), None);
assert_eq!(sniff_license(&dir.path().join("missing")), None);
}
}
+99
View File
@@ -0,0 +1,99 @@
//! Git handling for `pkh new`: initialize a repository in the scaffolded
//! tree unless it is already inside one (the `.gitignore`s are written
//! regardless).
use std::path::Path;
/// Ensure `dir` has a git repository when it should: when `dir` is already
/// inside a work tree (its own or a parent's), nothing is initialized and
/// `Ok(false)` is returned with an info log; otherwise a repository is
/// initialized in `dir` when `init` is set (the `--no-git` case passes
/// `init = false`).
pub fn ensure_repository(dir: &Path, init: bool) -> Result<bool, Box<dyn std::error::Error>> {
match git2::Repository::discover(dir) {
Ok(_) => {
log::info!(
"Already inside a git repository; skipping git init \
(the .gitignore files are written anyway)"
);
Ok(false)
}
Err(_) if init => {
git2::Repository::init(dir)?;
log::info!("Initialized empty git repository in {}", dir.display());
Ok(true)
}
// --no-git: gitignores only.
Err(_) => Ok(false),
}
}
/// Whether `dir` sits inside a git work tree (its own or a parent's), in the
/// spirit of `git rev-parse --is-inside-work-tree` (like
/// [`crate::new::origin::GitOrigin::detect`]). Fail-soft: when `git` cannot
/// be run or the probe fails, `dir` counts as outside — the caller keeps the
/// behavior it would have without the probe, and the scaffold-time
/// repository discovery in [`ensure_repository`] has the final word anyway.
pub fn inside_work_tree(dir: &Path) -> bool {
let Ok(output) = std::process::Command::new("git")
.args(["rev-parse", "--is-inside-work-tree"])
.current_dir(dir)
.output()
else {
return false;
};
output.status.success() && String::from_utf8_lossy(&output.stdout).trim() == "true"
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::tempdir;
#[test]
fn init_skipped_with_no_git() {
let dir = tempdir().unwrap();
assert!(!ensure_repository(dir.path(), false).unwrap());
assert!(!dir.path().join(".git").exists());
}
#[test]
fn init_creates_repository() {
let dir = tempdir().unwrap();
assert!(ensure_repository(dir.path(), true).unwrap());
assert!(dir.path().join(".git").exists());
// A second call discovers the fresh repository and skips init.
assert!(!ensure_repository(dir.path(), true).unwrap());
}
#[test]
fn parent_repository_is_discovered() {
let dir = tempdir().unwrap();
let sub = dir.path().join("sub");
std::fs::create_dir_all(&sub).unwrap();
git2::Repository::init(dir.path()).unwrap();
// The subdirectory is already inside the parent work tree.
assert!(!ensure_repository(&sub, true).unwrap());
assert!(!sub.join(".git").exists());
}
/// The probe answers like `git rev-parse --is-inside-work-tree`:
/// outside any repository it says no, inside one (at any depth, as a
/// skeleton mode scaffold would probe through its parent directory) it
/// says yes without touching the tree.
#[test]
fn inside_work_tree_follows_the_parent_repository() {
let dir = tempdir().unwrap();
assert!(!inside_work_tree(dir.path()));
git2::Repository::init(dir.path()).unwrap();
assert!(inside_work_tree(dir.path()));
let sub = dir.path().join("sub");
std::fs::create_dir(&sub).unwrap();
assert!(inside_work_tree(&sub));
assert!(!sub.join(".git").exists());
}
}
+224
View File
@@ -0,0 +1,224 @@
//! License reference data for `pkh new`, bundled as `data/licenses.yml`.
//!
//! The license knowledge of the scaffolder used to live in three places —
//! the wizard menu (`questions.rs`), the parse/SPDX mapping (`options.rs`)
//! and the license-file sniffing inputs (`detect.rs`) — kept in sync by
//! comments only. It is one table now: [`entries`] carries each curated
//! license's menu label, accepted spellings and detection markers,
//! [`detect_files`] the candidate license file names and [`url_template`]
//! the SPDX page URL template, so the lists cannot drift apart and adding
//! a license is a YAML entry.
//!
//! The `License` enum stays in `options.rs` (`NewOptions` and the template
//! rendering match on its variants); the enum's variants and the table's
//! ids are locked together by a consistency test there. The free-text
//! "Other (enter a SPDX identifier)" wizard entry is UX, not data, and
//! stays in Rust — like `License::Custom`'s code-driven parse path.
use serde::Deserialize;
use crate::data::embed_data;
/// One marker set of the license-text sniff: the set matches when every
/// marker of `all` occurs in the lowercased license text and none of
/// `unless` does (all-markers-within-a-list, any-marker-list semantics).
#[derive(Debug, Deserialize)]
pub(crate) struct DetectMarkerSet {
/// Substrings that must all occur in the license text
pub(crate) all: Vec<String>,
/// Substrings that must all be absent for the set to match
#[serde(default)]
pub(crate) unless: Vec<String>,
}
/// One curated license of the bundled table: everything `pkh new` knows
/// about it — identifier, menu label, accepted spellings and sniff markers
#[derive(Debug, Deserialize)]
pub(crate) struct LicenseEntry {
/// SPDX identifier: written to `debian/copyright`, returned by the
/// license sniff, substituted into [`url_template`]
pub(crate) id: String,
/// Label offered by the wizard license menu
pub(crate) menu: String,
/// Inputs accepted by `License::parse` (case-insensitive); must
/// include the id itself
pub(crate) spellings: Vec<String>,
/// Marker sets of the license-text sniff (see [`detect_from_text`])
pub(crate) detect_markers: Vec<DetectMarkerSet>,
}
/// The bundled license table (`data/licenses.yml`): the curated licenses
/// plus the shared sniffing inputs and the SPDX URL template
#[derive(Debug, Deserialize)]
struct LicensesData {
/// SPDX license page URL template (`{id}` placeholder)
license_url_template: String,
/// Candidate license file names of the sniff, in preference order
detect_files: Vec<String>,
/// The curated licenses, in wizard-menu order
licenses: Vec<LicenseEntry>,
}
embed_data! {
static ref LICENSES_DATA: LicensesData = "../../data/licenses.yml"
}
/// The curated license entries, in the order offered by the wizard menu
pub(crate) fn entries() -> &'static [LicenseEntry] {
&LICENSES_DATA.licenses
}
/// The entry whose `spellings` contain `input` (case-insensitive): the
/// table lookup behind `License::parse`'s curated arm
pub(crate) fn entry_for_spelling(input: &str) -> Option<&'static LicenseEntry> {
entries().iter().find(|entry| {
entry
.spellings
.iter()
.any(|s| s.eq_ignore_ascii_case(input))
})
}
/// The candidate license file names looked at by the license sniff, in
/// preference order, shared by every license
pub(crate) fn detect_files() -> impl Iterator<Item = &'static str> {
LICENSES_DATA.detect_files.iter().map(String::as_str)
}
/// The SPDX identifier of the (lowercased) license `text`, `None` when
/// nothing is recognizable: the first entry (menu order) with a matching
/// marker set wins. The caller lowercases the text; the markers are
/// lowercase substrings.
pub(crate) fn detect_from_text(text: &str) -> Option<&'static str> {
entries()
.iter()
.find(|entry| markers_match(entry, text))
.map(|entry| entry.id.as_str())
}
/// Whether any marker set of `entry` matches `text` (see
/// [`DetectMarkerSet`] for the semantics)
fn markers_match(entry: &LicenseEntry, text: &str) -> bool {
entry.detect_markers.iter().any(|set| {
set.all.iter().all(|marker| text.contains(marker))
&& set.unless.iter().all(|marker| !text.contains(marker))
})
}
/// The SPDX license page URL template of the table, carrying the license
/// identifier as an `{id}` placeholder
pub(crate) fn url_template() -> &'static str {
LICENSES_DATA.license_url_template.as_str()
}
#[cfg(test)]
mod tests {
use super::*;
/// Every entry is menu-ready: non-empty id/menu, at least one spelling
/// and one non-empty marker set, and no duplicate ids or menus (the
/// wizard select would silently shadow a duplicate menu label).
#[test]
fn entries_are_well_formed() {
assert!(!entries().is_empty());
let mut ids: Vec<&str> = Vec::new();
let mut menus: Vec<&str> = Vec::new();
for entry in entries() {
assert!(!entry.id.is_empty());
assert!(!entry.menu.is_empty());
assert!(!entry.spellings.is_empty());
assert!(
!entry.detect_markers.is_empty(),
"entry '{}' has no marker set",
entry.id
);
for set in &entry.detect_markers {
assert!(
!set.all.is_empty(),
"entry '{}' has an empty marker set",
entry.id
);
}
ids.push(entry.id.as_str());
menus.push(entry.menu.as_str());
}
ids.sort_unstable();
ids.dedup();
menus.sort_unstable();
menus.dedup();
assert_eq!(ids.len(), entries().len(), "duplicate ids");
assert_eq!(menus.len(), entries().len(), "duplicate menus");
}
/// The shared sniffing inputs and the URL template: the canonical file
/// names in preference order, and a template the spdx_url
/// substitution can render.
#[test]
fn sniff_inputs_and_url_template() {
let files: Vec<&str> = detect_files().collect();
assert!(!files.is_empty());
assert_eq!(files[0], "LICENSE");
assert!(files.contains(&"COPYING"));
assert!(url_template().starts_with("https://spdx.org/licenses/"));
assert!(url_template().contains("{id}"));
}
/// The marker semantics on hand-built texts: every `all` marker must
/// occur, every `unless` marker must not, any set of an entry
/// suffices, and the sets are mutually exclusive across entries —
/// the LGPL/GPL substring relation and the multi-license texts land
/// on the same entry the old hardcoded cascade picked.
#[test]
fn marker_sets_keep_the_old_cascade_results() {
// Nothing recognizable.
assert_eq!(detect_from_text("do whatever you want"), None);
// Version discrimination of the GPL family.
assert_eq!(
detect_from_text("gnu general public license\nversion 3"),
Some("GPL-3.0+")
);
assert_eq!(
detect_from_text("gnu general public license\nversion 2, june 1991"),
Some("GPL-2.0+")
);
// "lesser general public license" contains "general public
// license": LGPL texts must stay on their entries.
assert_eq!(
detect_from_text("gnu lesser general public license\nversion 3"),
Some("LGPL-3.0+")
);
assert_eq!(
detect_from_text("gnu lesser general public license\nversion 2.1"),
Some("LGPL-2.1+")
);
// A text naming both versions is the 2.1 wording (version 2.1
// wins), like the old unless-less branch pair did.
assert_eq!(
detect_from_text("gnu lesser general public license\nversion 3, like version 2.1"),
Some("LGPL-2.1+")
);
// Multi-license texts: the GPL/Apache reference is the stronger
// one, so MIT does not steal them.
assert_eq!(
detect_from_text("mit license\nunder the gnu general public license, version 2"),
Some("GPL-2.0+")
);
assert_eq!(
detect_from_text(
"permission is hereby granted, free of charge\ndual-licensed under the apache license version 2"
),
Some("Apache-2.0")
);
// BSD-2 vs BSD-3: the endorsement clause is the discriminator.
assert_eq!(
detect_from_text("redistribution and use in source and binary forms"),
Some("BSD-2-Clause")
);
assert_eq!(
detect_from_text(
"redistribution and use in source and binary forms\nmay be used to endorse or promote"
),
Some("BSD-3-Clause")
);
}
}
+935
View File
@@ -0,0 +1,935 @@
//! `pkh new`: interactive-first package scaffolding (see
//! `plans/pkh-new.md`).
//!
//! This module orchestrates a scaffold run: target directory checks, project
//! detection, in-memory rendering of every file (all-or-nothing write), the
//! root `.gitignore` merge (skeleton build artifacts plus the template's own
//! entries, e.g. the rust vendored layout), the template post-write hook
//! (e.g. `cargo vendor`), orig tarball creation
//! (from the origin the run decided on — see [`origin`] and [`orig`]), git
//! initialization, structural verification and the next-steps message.
//! The interactive wizard ([`questions`]) fills a [`options::NewCli`] from
//! its answers on a TTY and reuses [`options::resolve`] as the single source
//! of truth for defaults and validation; without a TTY the same resolution
//! runs flag-driven.
pub mod debian;
pub mod detect;
pub mod git;
/// License reference data for the scaffolder (bundled `data/licenses.yml`)
pub(crate) mod licenses;
pub mod options;
pub mod orig;
pub mod origin;
pub mod questions;
pub mod templates;
pub mod verify;
use std::error::Error;
use std::time::Duration;
use indicatif::{MultiProgress, ProgressBar, ProgressStyle};
use options::{NewOptions, SourceFormat};
use templates::{OutputFile, ScaffoldOutcome};
/// Scaffold a full Debian source tree from `opts`.
///
/// Steps, aborting early with a pointed error message:
/// 1. resolve the template from the registry,
/// 2. check the target directory (refuse an existing `debian/control`),
/// 3. render every file in memory and check for collisions,
/// 4. write the files all-or-nothing (plus the root `.gitignore` merge:
/// the skeleton build-artifact entries in skeleton mode and the
/// template's own entries — rust: the vendored layout — in every mode,
/// appending to an existing file),
/// 5. run the template post-write hook (e.g. `cargo vendor`, so the vendored
/// sources land inside the orig tarball created next),
/// 6. create the orig tarball (quilt only, refusing overwrites),
/// 7. `git init` unless `--no-git` or already inside a repository,
/// 8. run the structural verification,
/// 9. print the success message with the next steps.
///
/// On success the [`ScaffoldOutcome`] of the post-write hook is returned, so
/// the caller can adapt the end of the flow (e.g. the verification offer)
/// to what the templates managed to do.
pub fn scaffold(
opts: NewOptions,
multi: &MultiProgress,
) -> Result<ScaffoldOutcome, Box<dyn Error>> {
let pb = multi.add(ProgressBar::new_spinner());
pb.enable_steady_tick(Duration::from_millis(50));
pb.set_style(
ProgressStyle::default_bar()
.template("> {spinner:.blue} {prefix}")
.unwrap(),
);
pb.set_prefix("Scaffolding");
let result = scaffold_steps(&opts, &pb);
// Clear the spinner whatever the outcome; errors are reported by the
// caller as plain log lines.
pb.finish_and_clear();
multi.remove(&pb);
if let Ok(outcome) = &result {
print_success(&opts, outcome);
}
result
}
/// The scaffold steps proper, reporting progress through `pb`. Nothing is
/// written to the filesystem before every file rendered successfully.
fn scaffold_steps(opts: &NewOptions, pb: &ProgressBar) -> Result<ScaffoldOutcome, Box<dyn Error>> {
// 1. Template resolution: an id without a registered template fails
// here with the friendly message instead of a parse error.
let template = templates::get(opts.template).ok_or_else(|| {
format!(
"The '{}' template has no registered implementation. \
This is a pkh bug; please report it.",
opts.template,
)
})?;
// 2. Target directory checks.
let cwd = std::env::current_dir()?;
let target = opts.target_dir(&cwd);
if target.join("debian/control").exists() {
return Err(format!(
"'{}' already contains a debian/control file: pkh new refuses to \
touch an existing Debian packaging tree",
target.display()
)
.into());
}
match &opts.source_dir {
options::SourceDir::Skeleton => {
if target.exists() {
if !target.is_dir() {
return Err(
format!("'{}' exists and is not a directory", target.display()).into(),
);
}
if std::fs::read_dir(&target)?.next().is_some() {
return Err(format!(
"directory '{}' already exists and is not empty: \
pkh new refuses to scaffold into it",
target.display()
)
.into());
}
}
}
options::SourceDir::Here => {
// The cwd always exists.
}
options::SourceDir::Path(path) => {
if !path.is_dir() {
return Err(format!(
"source directory '{}' does not exist or is not a directory",
path.display()
)
.into());
}
}
}
// Fail before writing anything when the orig tarball already exists.
if opts.source_format == SourceFormat::Quilt
&& let Some(tarball) =
debian::orig_tarball_path(&target, &opts.name, &opts.upstream_version_no_epoch())
&& tarball.exists()
{
return Err(format!(
"'{}' already exists: pkh new refuses to overwrite it. \
Remove it first, or pass --native to skip the orig tarball.",
tarball.display()
)
.into());
}
// 3. Render everything in memory, then check for collisions (within the
// generated set and against existing files).
pb.set_message("Rendering files");
let skeleton = matches!(opts.source_dir, options::SourceDir::Skeleton);
let mut files: Vec<OutputFile> = debian::files(opts, template);
if skeleton {
files.extend(template.skeleton(opts));
}
files.extend(template.debian(opts));
let paths: Vec<String> = files.iter().map(|f| f.path.clone()).collect();
options::check_file_collisions(&paths)?;
for path in &paths {
let existing = target.join(path);
if existing.exists() {
return Err(format!(
"refusing to overwrite existing file '{}'",
existing.display()
)
.into());
}
}
// 4. Write the files (all-or-nothing: nothing was written on any error
// above).
pb.set_message("Writing files");
debian::write_files(&target, &files)?;
// Root .gitignore: the skeleton build-artifact entries in skeleton
// mode, plus the template's own entries (rust: the vendored layout) in
// every mode — missing ones appended, an existing file never
// overwritten.
let template_gitignore = template.gitignore_entries();
let mut entries: Vec<&str> = Vec::new();
let mut header = None;
if skeleton {
entries.extend(debian::ROOT_GITIGNORE_ENTRIES);
header = Some(debian::ROOT_GITIGNORE_HEADER);
}
entries.extend(template_gitignore.iter().map(String::as_str));
if !entries.is_empty()
&& let Some(contents) = debian::merge_gitignore_entries(
std::fs::read_to_string(target.join(".gitignore"))
.ok()
.as_deref(),
&entries,
header,
)
{
std::fs::write(target.join(".gitignore"), contents)?;
}
// 5. Template post-write hook: run before the orig tarball is created,
// so files added here (rust: vendor/ + .cargo/config.toml) land
// inside it (or inside the orig-vendor component). The outcome
// (e.g. a failed vendoring) is threaded back to the caller.
pb.set_message("Running template hooks");
let mut outcome = template.post_write(opts, &target)?;
// 6. Orig tarball (quilt only), from the origin the run decided on.
// A vendored rust tree gets its `vendor/` directory moved into the
// separate dpkg upstream component `orig-vendor`, regenerable
// independently of the upstream sources (native packages have no
// orig at all: vendor/ simply lives in the tree).
if opts.source_format == SourceFormat::Quilt {
pb.set_message("Creating orig tarball");
let vendored_rust =
template.id() == options::TemplateId::RUST && orig::has_vendored_dir(&target);
let created = orig::create_orig(
&target,
&opts.name,
&opts.upstream_version_no_epoch(),
opts.orig
.as_ref()
.ok_or("internal error: a quilt scaffold needs an orig-tarball plan")?,
vendored_rust,
)?;
outcome.orig_origin = Some(created.label);
if vendored_rust {
pb.set_message("Creating the orig-vendor component");
orig::create_vendor_component(&target, &opts.name, &opts.upstream_version_no_epoch())?;
}
}
// 7. Git.
pb.set_message("Initializing git");
git::ensure_repository(&target, opts.git)?;
// 8. Structural verification.
pb.set_message("Verifying");
verify::verify(&target)?;
Ok(outcome)
}
/// The success message: what was created and the next steps.
fn print_success(opts: &NewOptions, outcome: &ScaffoldOutcome) {
let target =
opts.target_dir(&std::env::current_dir().unwrap_or_else(|_| std::path::PathBuf::from(".")));
// `display_path` yields an empty string when the target is the cwd
// itself (Here mode): `Created .` would be cryptic, so spell the
// location out; the skeleton/path modes keep the `<dir>` display.
let display = crate::report::display_path(&target);
let location = if display.is_empty() {
"package in the current directory".to_string()
} else {
display.clone()
};
log::info!(
"Created {location} — {} ({}-{}) for {}/{}, template '{}'",
opts.name,
opts.upstream_version,
opts.revision,
opts.dist,
opts.series,
opts.template
);
if let Some(orig_origin) = &outcome.orig_origin {
log::info!("Orig tarball: {orig_origin}");
}
log::info!("Next steps:");
log::info!(" cd {}", if display.is_empty() { "." } else { &display });
if opts.release {
log::info!(
" pkh chlog # for later changes; the entry already targets {}",
opts.series
);
} else {
log::info!(
" pkh chlog # releases the UNRELEASED entry to '{}' when ready",
opts.series
);
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::new::options::{License, OrigOrigin, SourceDir, TemplateId};
use serial_test::serial;
use tempfile::tempdir;
fn opts(template: TemplateId, name: &str, source_dir: SourceDir) -> NewOptions {
// Mirror the resolve() derivation: skeletons are native by default,
// existing projects quilt with a working-tree snapshot orig.
let (source_format, orig) = match source_dir {
SourceDir::Skeleton => (SourceFormat::Native, None),
SourceDir::Here | SourceDir::Path(_) => {
(SourceFormat::Quilt, Some(OrigOrigin::Snapshot))
}
};
NewOptions {
name: name.to_string(),
template,
source_dir,
upstream_version: "0.1.0".into(),
revision: 1,
summary: "A tool that does one thing well".into(),
long_description: "A tool that does one thing well".into(),
homepage: None,
license: License::Mit,
command: name.to_string(),
maintainer: ("Jane Doe".into(), "jane@example.com".into()),
dist: "ubuntu".into(),
series: "resolute".into(),
release: false,
depends: Vec::new(),
source_format,
orig,
git: false,
autopkgtest: false,
pkg_config: false,
watch: None,
}
}
/// Run `scaffold` with the cwd changed to `dir` (restored afterwards);
/// must run under `#[serial]` because the cwd is process-global.
fn scaffold_in(
dir: &std::path::Path,
opts: NewOptions,
) -> Result<ScaffoldOutcome, Box<dyn Error>> {
let previous = std::env::current_dir()?;
std::env::set_current_dir(dir)?;
// Hidden draw target: in tests the spinner would redraw from its
// steady-tick thread straight to the real stderr
let result = scaffold(
opts,
&MultiProgress::with_draw_target(crate::ui::progress_draw_target()),
);
std::env::set_current_dir(previous)?;
result
}
#[test]
#[serial]
fn scaffold_shell_skeleton_tree() {
let dir = tempdir().unwrap();
scaffold_in(
dir.path(),
opts(TemplateId::SHELL, "mytool", SourceDir::Skeleton),
)
.unwrap();
let tree = dir.path().join("mytool");
// Every expected file exists.
for path in [
"debian/control",
"debian/changelog",
"debian/rules",
"debian/copyright",
"debian/source/format",
"debian/.gitignore",
"debian/install",
"mytool.sh",
".gitignore",
] {
assert!(tree.join(path).exists(), "{path} missing");
}
// rules and the script carry the exec bit.
use std::os::unix::fs::PermissionsExt;
for executable in ["debian/rules", "mytool.sh"] {
let mode = std::fs::metadata(tree.join(executable))
.unwrap()
.permissions()
.mode();
assert_eq!(mode & 0o777, 0o755, "{executable}");
}
// control re-parses.
let control = crate::debian::ControlInfo::parse(&tree.join("debian/control")).unwrap();
assert_eq!(control.source_name(), "mytool");
assert_eq!(control.binaries[0].get("Architecture"), Some("all"));
assert_eq!(
control.source.get("Build-Depends"),
Some("debhelper-compat (= 13)")
);
assert!(control.binaries[0].get("Depends").is_none());
// changelog re-parses: UNRELEASED by default.
let (_, version, distribution) =
crate::changelog::parse_changelog_header(&tree.join("debian/changelog")).unwrap();
assert_eq!(version, "0.1.0-1");
assert_eq!(distribution, "UNRELEASED");
// A fresh skeleton is 3.0 (native) by default: no local-options and
// no orig tarball anywhere.
assert_eq!(
std::fs::read_to_string(tree.join("debian/source/format")).unwrap(),
"3.0 (native)\n"
);
assert!(!tree.join("debian/source/local-options").exists());
assert!(!dir.path().join("mytool_0.1.0.orig.tar.xz").exists());
// install mapping.
assert_eq!(
std::fs::read_to_string(tree.join("debian/install")).unwrap(),
"mytool.sh usr/bin/mytool\n"
);
// Root .gitignore.
let gitignore = std::fs::read_to_string(tree.join(".gitignore")).unwrap();
assert!(gitignore.contains("*.deb"));
assert!(gitignore.contains("target/"));
}
/// A skeleton forced to quilt keeps the snapshot behavior: orig tarball
/// with the skeleton files, `debian/` excluded, local-options present.
#[test]
#[serial]
fn scaffold_skeleton_forced_quilt_snapshots_the_tree() {
let dir = tempdir().unwrap();
let mut o = opts(TemplateId::SHELL, "mytool", SourceDir::Skeleton);
o.source_format = SourceFormat::Quilt;
o.orig = Some(OrigOrigin::Snapshot);
scaffold_in(dir.path(), o).unwrap();
let tree = dir.path().join("mytool");
assert_eq!(
std::fs::read_to_string(tree.join("debian/source/format")).unwrap(),
"3.0 (quilt)\n"
);
assert_eq!(
std::fs::read_to_string(tree.join("debian/source/local-options")).unwrap(),
"single-debian-patch\n\
extend-diff-ignore = ^target/\n\
extend-diff-ignore = ^node_modules/\n\
extend-diff-ignore = ^\\.venv/\n"
);
// Orig tarball: contains the skeleton file, excludes debian/.
let tarball = dir.path().join("mytool_0.1.0.orig.tar.xz");
assert!(tarball.exists());
let mut archive = tar::Archive::new(xz2::read::XzDecoder::new(
std::fs::File::open(&tarball).unwrap(),
));
let names: Vec<String> = archive
.entries()
.unwrap()
.map(|entry| {
entry
.unwrap()
.path()
.unwrap()
.to_string_lossy()
.into_owned()
})
.collect();
assert!(
names.iter().any(|n| n == "mytool-0.1.0/mytool.sh"),
"{names:?}"
);
assert!(!names.iter().any(|n| n.contains("debian")), "{names:?}");
}
#[test]
#[serial]
fn scaffold_empty_base_and_metapackage_flavors() {
let dir = tempdir().unwrap();
// Metapackage flavor: non-empty depends.
let mut o = opts(TemplateId::EMPTY, "metapkg", SourceDir::Skeleton);
o.depends = vec!["hello".into(), "hello-data (>= 1.0)".into()];
scaffold_in(dir.path(), o).unwrap();
let tree = dir.path().join("metapkg");
let control = crate::debian::ControlInfo::parse(&tree.join("debian/control")).unwrap();
assert_eq!(
control.binaries[0].get("Depends"),
Some("hello,\nhello-data (>= 1.0)")
);
assert_eq!(control.binaries[0].get("Architecture"), Some("all"));
// No install file, no build-system skeleton: the README stub only.
// A native skeleton carries no orig tarball: the README simply
// lives in the tree.
assert!(!tree.join("debian/install").exists());
assert!(tree.join("README").exists());
assert!(!dir.path().join("metapkg_0.1.0.orig.tar.xz").exists());
// Empty base flavor: no depends, no Depends field.
let dir = tempdir().unwrap();
scaffold_in(
dir.path(),
opts(TemplateId::EMPTY, "basepkg", SourceDir::Skeleton),
)
.unwrap();
let control =
crate::debian::ControlInfo::parse(&dir.path().join("basepkg/debian/control")).unwrap();
assert!(control.binaries[0].get("Depends").is_none());
}
#[test]
#[serial]
fn scaffold_release_targets_series() {
let dir = tempdir().unwrap();
let mut o = opts(TemplateId::EMPTY, "released", SourceDir::Skeleton);
o.release = true;
scaffold_in(dir.path(), o).unwrap();
let (_, _, distribution) =
crate::changelog::parse_changelog_header(&dir.path().join("released/debian/changelog"))
.unwrap();
assert_eq!(distribution, "resolute");
}
#[test]
#[serial]
fn scaffold_here_mode_packages_existing_dir() {
let dir = tempdir().unwrap();
// Here mode packages the cwd itself, so the orig tarball lands one
// level up (dpkg convention): package a subdirectory of the tempdir
// to keep the artifacts inside it.
let tree = dir.path().join("packdir");
std::fs::create_dir_all(&tree).unwrap();
std::fs::write(tree.join("run.sh"), "#!/bin/sh\necho hi\n").unwrap();
scaffold_in(&tree, opts(TemplateId::SHELL, "runtool", SourceDir::Here)).unwrap();
// debian/ lands directly in the directory; no skeleton file, no
// root .gitignore (the shell template contributes none and the
// artifact entries are skeleton-only), no debian/install (the
// generated one would reference the non-existent skeleton script),
// and the existing script is left alone.
assert!(tree.join("debian/control").exists());
assert!(!tree.join("runtool.sh").exists());
assert!(!tree.join(".gitignore").exists());
assert!(!tree.join("debian/install").exists());
assert_eq!(
std::fs::read_to_string(tree.join("run.sh")).unwrap(),
"#!/bin/sh\necho hi\n"
);
// The orig tarball carries the pre-existing script, next to the tree.
let tarball = dir.path().join("runtool_0.1.0.orig.tar.xz");
let mut archive = tar::Archive::new(xz2::read::XzDecoder::new(
std::fs::File::open(&tarball).unwrap(),
));
let names: Vec<String> = archive
.entries()
.unwrap()
.map(|entry| {
entry
.unwrap()
.path()
.unwrap()
.to_string_lossy()
.into_owned()
})
.collect();
assert!(
names.iter().any(|n| n == "runtool-0.1.0/run.sh"),
"{names:?}"
);
}
#[test]
#[serial]
fn scaffold_refuses_existing_trees_and_artifacts() {
let dir = tempdir().unwrap();
// Existing debian/control.
let tree = dir.path().join("mytool");
std::fs::create_dir_all(tree.join("debian")).unwrap();
std::fs::write(tree.join("debian/control"), "Source: mytool\n").unwrap();
let err = scaffold_in(
dir.path(),
opts(TemplateId::SHELL, "mytool", SourceDir::Skeleton),
)
.unwrap_err();
assert!(err.to_string().contains("debian/control"), "{err}");
// Non-empty skeleton target.
let dir = tempdir().unwrap();
std::fs::create_dir_all(dir.path().join("mytool")).unwrap();
std::fs::write(dir.path().join("mytool/junk"), "x").unwrap();
let err = scaffold_in(
dir.path(),
opts(TemplateId::SHELL, "mytool", SourceDir::Skeleton),
)
.unwrap_err();
assert!(err.to_string().contains("not empty"), "{err}");
// Existing orig tarball (quilt only): nothing gets written.
let dir = tempdir().unwrap();
std::fs::write(dir.path().join("mytool_0.1.0.orig.tar.xz"), b"old").unwrap();
let mut o = opts(TemplateId::SHELL, "mytool", SourceDir::Skeleton);
o.source_format = SourceFormat::Quilt;
o.orig = Some(OrigOrigin::Snapshot);
let err = scaffold_in(dir.path(), o).unwrap_err();
assert!(err.to_string().contains("already exists"), "{err}");
assert!(!dir.path().join("mytool/debian/control").exists());
// Missing --source directory.
let dir = tempdir().unwrap();
let err = scaffold_in(
dir.path(),
opts(
TemplateId::SHELL,
"mytool",
SourceDir::Path(dir.path().join("missing")),
),
)
.unwrap_err();
assert!(err.to_string().contains("does not exist"), "{err}");
}
/// The skeleton default is `3.0 (native)`: no orig tarball, no
/// `debian/source/local-options`.
#[test]
#[serial]
fn scaffold_skeleton_defaults_to_native() {
let dir = tempdir().unwrap();
scaffold_in(
dir.path(),
opts(TemplateId::SHELL, "nativepkg", SourceDir::Skeleton),
)
.unwrap();
let tree = dir.path().join("nativepkg");
assert!(!dir.path().join("nativepkg_0.1.0.orig.tar.xz").exists());
assert!(!tree.join("debian/source/local-options").exists());
assert_eq!(
std::fs::read_to_string(tree.join("debian/source/format")).unwrap(),
"3.0 (native)\n"
);
}
/// End-to-end rust skeleton: the vendoring hook runs over the tree
/// (native format: everything simply lives in the tree, no orig
/// tarball). The vendoring step needs host cargo; on a cargo-less host
/// the scaffold still succeeds with a warning and a `vendoring_failed`
/// outcome. Keyed against the `RUSTUP_TOOLCHAIN` tests of the rust
/// template: they mutate the process-global environment the cargo shim
/// would pick up mid-vendoring.
#[test]
#[serial]
#[serial(RUSTUP_TOOLCHAIN)]
fn scaffold_rust_skeleton_vendors_into_the_tree() {
let dir = tempdir().unwrap();
let outcome = scaffold_in(
dir.path(),
opts(TemplateId::RUST, "mytool", SourceDir::Skeleton),
)
.unwrap();
let has_cargo = crate::new::templates::find_on_path("cargo").is_some();
assert_eq!(outcome.vendoring_failed, !has_cargo);
// Native skeleton: no orig tarball at all.
assert_eq!(outcome.orig_origin, None);
assert!(!dir.path().join("mytool_0.1.0.orig.tar.xz").exists());
let tree = dir.path().join("mytool");
assert!(tree.join("Cargo.toml").exists());
assert!(tree.join("src/main.rs").exists());
// The vendoring step is what creates Cargo.lock for a skeleton.
assert_eq!(tree.join("Cargo.lock").exists(), has_cargo);
// rules: the vendored build overrides, and `--locked` exactly when
// the vendoring step left a lockfile behind (it appears after the
// rules were rendered, so the hook patches it in).
let rules = std::fs::read_to_string(tree.join("debian/rules")).unwrap();
assert!(rules.contains("%:\n\tdh $@\n"));
assert!(rules.contains("override_dh_auto_build:\n\tcargo build --release --offline"));
assert!(rules.contains("override_dh_auto_install:\n\tinstall -Dm755 target/release/mytool debian/mytool/usr/bin/mytool"));
assert_eq!(rules.contains("--locked"), has_cargo);
// control: Architecture any + the cargo/rustc build-deps.
let control = crate::debian::ControlInfo::parse(&tree.join("debian/control")).unwrap();
assert_eq!(control.binaries[0].get("Architecture"), Some("any"));
assert_eq!(
control.source.get("Build-Depends"),
Some("debhelper-compat (= 13),\ncargo:native,\nrustc:native")
);
// The offline config exists in the tree when host cargo vendored
// the skeleton.
if has_cargo {
let config = std::fs::read_to_string(tree.join(".cargo/config.toml")).unwrap();
assert!(config.contains("[source.crates-io]"), "{config}");
assert!(config.contains("[net]\noffline = true"), "{config}");
}
// Root .gitignore: the skeleton build-artifact entries plus the
// template's vendoring entries (contributed in every mode).
let gitignore = std::fs::read_to_string(tree.join(".gitignore")).unwrap();
assert!(gitignore.contains("# pkh build artifacts"), "{gitignore}");
assert!(gitignore.contains("*.deb"), "{gitignore}");
assert!(gitignore.contains("target/"), "{gitignore}");
assert!(gitignore.contains("vendor/\n"), "{gitignore}");
assert!(gitignore.contains(".cargo/config.toml\n"), "{gitignore}");
}
/// The rust template's vendoring entries land in the root `.gitignore`
/// in every mode: a Here-mode tree gets them appended to its existing
/// file (custom lines kept, no header comment), while the skeleton-only
/// build-artifact entries do not appear.
#[test]
#[serial]
fn scaffold_rust_here_merges_vendoring_gitignore_entries() {
let dir = tempdir().unwrap();
let tree = dir.path().join("packdir");
std::fs::create_dir_all(&tree).unwrap();
std::fs::write(tree.join(".gitignore"), "# my project\n*.log\n").unwrap();
scaffold_in(&tree, opts(TemplateId::RUST, "mytool", SourceDir::Here)).unwrap();
let gitignore = std::fs::read_to_string(tree.join(".gitignore")).unwrap();
assert!(
gitignore.starts_with("# my project\n*.log\n"),
"{gitignore}"
);
assert!(gitignore.contains("vendor/\n"), "{gitignore}");
assert!(gitignore.contains(".cargo/config.toml\n"), "{gitignore}");
// The build-artifact entries stay skeleton-only.
assert!(!gitignore.contains("*.deb"), "{gitignore}");
assert!(!gitignore.contains("# pkh build artifacts"), "{gitignore}");
}
/// End-to-end vendored rust quilt package: the vendoring hook creates
/// `vendor/`, the main orig excludes it, the dpkg upstream component
/// `orig-vendor` carries it, and the real source build
/// (`run_source_build`) lists BOTH tarballs in the `.dsc` and succeeds.
/// Needs host cargo with a working crates.io sync (skipped gracefully
/// when either is unavailable) and the local dpkg tools.
#[test]
#[serial]
#[serial(RUSTUP_TOOLCHAIN)]
fn scaffold_rust_quilt_with_deps_vendors_into_a_component() {
let dir = tempdir().unwrap();
let source = dir.path().join("mytool");
std::fs::create_dir_all(source.join("src")).unwrap();
// `libc` resolves from the host's cargo registry cache; vendoring
// it needs one crates.io index sync.
std::fs::write(
source.join("Cargo.toml"),
"[package]\n\
name = \"mytool\"\n\
version = \"0.1.0\"\n\
edition = \"2021\"\n\
\n\
[dependencies]\n\
libc = \"0.2\"\n",
)
.unwrap();
std::fs::write(source.join("src/main.rs"), "fn main() {}\n").unwrap();
let mut o = opts(TemplateId::RUST, "mytool", SourceDir::Path(source.clone()));
o.source_format = SourceFormat::Quilt;
o.orig = Some(OrigOrigin::Snapshot);
let outcome = scaffold_in(dir.path(), o).unwrap();
if crate::new::templates::find_on_path("cargo").is_none() || outcome.vendoring_failed {
// No cargo on this host or the crates.io sync failed: the
// vendoring guarantees of this test cannot hold.
log::warn!("cargo/crates.io unavailable; skipping the vendored component checks");
return;
}
assert_eq!(
outcome.orig_origin,
Some("working tree snapshot".to_string())
);
// The main orig excludes vendor/ but carries the upstream files.
let tarball = dir.path().join("mytool_0.1.0.orig.tar.xz");
let mut archive = tar::Archive::new(xz2::read::XzDecoder::new(
std::fs::File::open(&tarball).unwrap(),
));
let names: Vec<String> = archive
.entries()
.unwrap()
.map(|entry| {
entry
.unwrap()
.path()
.unwrap()
.to_string_lossy()
.into_owned()
})
.collect();
assert!(
names.iter().any(|n| n == "mytool-0.1.0/Cargo.toml"),
"{names:?}"
);
assert!(!names.iter().any(|n| n.contains("vendor")), "{names:?}");
// The component carries vendor/ under a top-level vendor/ dir.
let component = dir.path().join("mytool_0.1.0.orig-vendor.tar.xz");
assert!(component.exists());
let mut archive = tar::Archive::new(xz2::read::XzDecoder::new(
std::fs::File::open(&component).unwrap(),
));
let names: Vec<String> = archive
.entries()
.unwrap()
.map(|entry| {
entry
.unwrap()
.path()
.unwrap()
.to_string_lossy()
.into_owned()
})
.collect();
assert!(
names.iter().any(|n| n.starts_with("vendor/libc/")),
"{names:?}"
);
// The real source build: the .dsc references both tarballs.
let output = crate::build::run_source_build(
&source,
&crate::build::SourceBuildOptions::default(),
&crate::report::Quiet,
)
.unwrap();
let dsc = std::fs::read_to_string(&output.dsc).unwrap();
assert!(dsc.contains("mytool_0.1.0.orig.tar.xz"), "{dsc}");
assert!(dsc.contains("mytool_0.1.0.orig-vendor.tar.xz"), "{dsc}");
assert!(
output
.tarballs
.iter()
.any(|t| t.ends_with("mytool_0.1.0.orig.tar.xz"))
);
assert!(
output
.tarballs
.iter()
.any(|t| t.ends_with("mytool_0.1.0.orig-vendor.tar.xz"))
);
assert!(!output.signed);
}
/// End-to-end python skeleton: pyproject-based Build-Depends (native
/// skeleton: the upstream files live in the tree, no orig tarball).
#[test]
#[serial]
fn scaffold_python_skeleton_tree() {
let dir = tempdir().unwrap();
scaffold_in(
dir.path(),
opts(TemplateId::PYTHON, "mytool", SourceDir::Skeleton),
)
.unwrap();
let tree = dir.path().join("mytool");
let control = crate::debian::ControlInfo::parse(&tree.join("debian/control")).unwrap();
assert_eq!(control.binaries[0].get("Architecture"), Some("all"));
assert_eq!(
control.source.get("Build-Depends"),
Some(
"debhelper-compat (= 13),\ndh-python,\npython3-all,\n\
pybuild-plugin-pyproject,\npython3-setuptools"
)
);
let rules = std::fs::read_to_string(tree.join("debian/rules")).unwrap();
assert!(rules.contains("%:\n\tdh $@ --with python3 --buildsystem=pybuild\n"));
assert!(tree.join("pyproject.toml").exists());
assert!(tree.join("mytool/__init__.py").exists());
assert!(!dir.path().join("mytool_0.1.0.orig.tar.xz").exists());
}
/// End-to-end: a quilt tree (Here mode over an existing source) passes
/// the real source build (`dpkg-source` and friends, same prerequisites
/// as the differential tests).
#[test]
#[serial]
fn scaffold_then_source_build_produces_artifacts() {
let dir = tempdir().unwrap();
let tree = dir.path().join("mytool");
std::fs::create_dir_all(&tree).unwrap();
std::fs::write(tree.join("run.sh"), "#!/bin/sh\necho hi\n").unwrap();
scaffold_in(&tree, opts(TemplateId::SHELL, "mytool", SourceDir::Here)).unwrap();
let output = crate::build::run_source_build(
&dir.path().join("mytool"),
&crate::build::SourceBuildOptions::default(),
&crate::report::Quiet,
)
.unwrap();
assert!(output.dsc.exists(), "{:?} missing", output.dsc);
assert!(output.buildinfo.exists(), "{:?} missing", output.buildinfo);
assert!(output.changes.exists(), "{:?} missing", output.changes);
// 3.0 (quilt): the orig tarball plus the debian diff tarball that
// dpkg-source generates for the debian/ directory.
assert_eq!(output.tarballs.len(), 2, "{:?}", output.tarballs);
assert!(output.tarballs[0].exists());
assert!(output.tarballs[1].exists());
// UNRELEASED: nothing is signed.
assert!(!output.signed);
}
/// End-to-end native: a self-authored skeleton (3.0 (native), no orig
/// tarball) builds into a .dsc without any tarball at all.
#[test]
#[serial]
fn scaffold_native_then_source_build_needs_no_tarball() {
let dir = tempdir().unwrap();
scaffold_in(
dir.path(),
opts(TemplateId::SHELL, "mytool", SourceDir::Skeleton),
)
.unwrap();
let output = crate::build::run_source_build(
&dir.path().join("mytool"),
&crate::build::SourceBuildOptions::default(),
&crate::report::Quiet,
)
.unwrap();
assert!(output.dsc.exists(), "{:?} missing", output.dsc);
assert!(output.changes.exists(), "{:?} missing", output.changes);
// 3.0 (native): one self-contained source tarball (debian/ inside),
// but no ORIG tarball.
assert_eq!(output.tarballs.len(), 1, "{:?}", output.tarballs);
assert!(
!output.tarballs[0]
.file_name()
.is_some_and(|name| name.to_string_lossy().contains("orig")),
"{:?}",
output.tarballs
);
}
}
+1646
View File
File diff suppressed because it is too large Load Diff
+1648
View File
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More