Compare commits

...
26 Commits
Author SHA1 Message Date
vhaudiquet e7b35f5c37 deps: treat corrupt Provides as undecidable, not unmet
CI / build (push) Failing after 2m58s
CI / test (push) Skipped
CI / snap (push) Skipped
A versioned Provides whose version failed to parse was silently
skipped, so a corrupt dpkg status entry could yield a wrong 'unmet'
verdict where the truth is 'cannot decide': unparseable provided
versions now set lackinfos like unparseable installed versions do.
Provides alternatives with a non-= constraint are likewise rejected as
a whole field (dpkg rejects the entry), replacing the skip-per-
alternative behavior that contradicted the code's own comment.
2026-09-17 20:08:20 +02:00
vhaudiquet 54274e9079 version: reject an empty debian revision
DebianVersion::parse accepted '1.0-' (empty revision after rsplit on
the last hyphen), where dpkg rejects it with 'revision number is
empty'; downstream filename construction produced garbage like
'foo_1.0-.dsc'. Keep the start-digit warning-only semantics of dpkg
(no new check there) and the accepted '1.0--1' split.
2026-09-17 19:52:32 +02:00
vhaudiquet 93296c26c1 checksums: add SHA-512 and a Checksums-* field parser
The checksum model only carried md5/sha1/sha256 while deb-buildinfo(5)
defines Checksums-Sha512, and there was no way to parse a Checksums-*
field body back into entries. Add ChecksumKind::Sha512 (computed
alongside the others), a field parser validating the
'<hex> <size> <name>' grammar, and an only-if-populated
Checksums-Sha512 emission in .buildinfo — deliberately dormant in the
dpkg-parity flows, which never emit it, and .dsc/.changes untouched.
2026-09-17 19:44:54 +02:00
vhaudiquet d6bad9fbbe debian: make Paragraph::set drop case-insensitive duplicates
set only replaced the first match and appended otherwise, so a
paragraph holding both 'Depends:' and 'depends:' kept a stale second
value after an update, silently re-emitted on serialization. set now
updates the first match in place and removes any other case-insensitive
duplicate; the parser stays lenient and keeps duplicates reachable via
iter().
2026-09-17 19:16:28 +02:00
vhaudiquet 607711a6b5 build: surface a panicked output-reader thread
run_command_capturing discarded the pump threads' join results: a
reader that died mid-capture (UI sink or log writer failing) reported a
successful build with truncated captured logs. A reader panic now fails
the command; when the child itself failed first, its error keeps
precedence and the reader panic is logged so the truncated output is
not silently lost.
2026-09-17 19:12:35 +02:00
vhaudiquet 47b462ad61 build: merge inherited DEB_BUILD_OPTIONS instead of overwriting it
The source-build pipeline exported its computed DEB_BUILD_OPTIONS
verbatim, silently dropping options the user set in the environment
(e.g. terse) where dpkg-buildpackage prepends the inherited value.
Options are now merged inherited-first through a shared helper, with
whitespace normalized.
2026-09-17 19:05:15 +02:00
vhaudiquet fa39851783 build: check stat's exit status and honor DPKG_ORIGINS_DIR
hashes_in_context never checked stat's exit status and parsed its size
with unwrap_or(0), silently recording zero-size artifacts in the
generated .changes/.buildinfo; stat failures and unparsable sizes are
now errors naming the file. current_vendor hardcoded
/etc/dpkg/origins/default while dpkg honors DPKG_ORIGINS_DIR (already
in the file's own ENV_ALLOWED list); the origins default is now
resolved against it with the usual fallback.
2026-09-17 18:56:32 +02:00
vhaudiquet 22e43741f3 build: make build_source_package(None) default to the current directory
None mapped to Path::new("."), whose parent is the empty string:
the output-directory derivation then always failed with 'cannot
determine output directory', making the documented Option default a
guaranteed-failure trap. Resolve None to the process's absolute
current working directory instead.
2026-09-17 18:50:09 +02:00
vhaudiquet 36875513ee build: stop turning read failures into silently wrong metadata
Two read_file(...).unwrap_or_default() calls masqueraded IO errors as
empty data: an unreadable debian/files became 'binary build with no
binary artifacts found; cannot distribute', and an unreadable dpkg
status file produced an empty Installed-Build-Depends. Tolerate a
missing debian/files (first build in a fresh tree) but propagate real
read errors, and hard-error on an unreadable status file like the
source-build path does. installed_build_depends_from_content also
returned a bare newline for zero entries, defeating render_buildinfo's
empty-guard and emitting a malformed 'Installed-Build-Depends:' field
with a blank continuation; it now returns an empty string so the field
is omitted.
2026-09-17 18:44:33 +02:00
vhaudiquet 06e591c665 build: fail binary-only metadata when the previous version is unparseable
The binNMU path swallowed parse errors with a let-chain: a changelog
that could not yield the previous version silently produced a .changes
with plain 'Source: pkg', no Binary-Only-Changes and no redistributed
previous .dsc. Propagate the parse error like the source-build path
does (a single-entry changelog stays tolerated), and reuse the
changelog already read instead of reading the file a second time.
2026-09-17 18:31:31 +02:00
vhaudiquet 12828f8498 deps: resolve :native against DEB_BUILD_ARCH in cross builds
CheckOpts had no build-arch concept: the build-side facts and :native
qualifiers resolved against the host arch, so in a cross build
(-a armhf on amd64) 'Build-Depends: foo:native' looked for an armhf
package where dpkg-checkbuilddeps looks for an amd64 one. CheckOpts
gains build_arch (DEB_BUILD_ARCH), used for :native and the dpkg status
attribution; bracketed arch restrictions keep evaluating against the
host arch.
2026-09-17 18:17:08 +02:00
vhaudiquet eaf1b40369 new: stream orig downloads, reap children on every path
The release-tarball download capped the whole request at 30 s (large
tarballs on slow links always failed and fell through to worse origins)
and buffered the entire body in memory: keep a 10 s connect timeout
only and stream the body to the temp file. The bzip2 -dc child of a
failing repack was neither killed nor waited on (zombie + open pipe);
it is now reaped on both paths. git archive no longer pipes a stderr
nobody drains (a chatty git deadlocked the archive) and any failure
after the destination file was created removes the empty or partial
tarball.
2026-09-17 18:03:24 +02:00
vhaudiquet 27ab4cb9ad put: fix ssh_config negation semantics and file precedence
Host pattern lists were evaluated per-pattern with 'any', so
'Host * !*.launchpad.net' matched ppa.launchpad.net via the wildcard;
a block now applies only if a positive pattern matches and no negated
one does (OpenSSH's rule). The system ssh_config was read first with
first-obtained-wins, inverting OpenSSH's user-over-system precedence;
the user file is read first now. A Match block also no longer leaks
the previous Host block's match state (its options are ignored until
the next Host).
2026-09-17 17:52:17 +02:00
vhaudiquet 3501096107 new: keep flat-tarball entries when repacking the orig tarball
The repack stripped the first path component of every entry, assuming a
single top-level directory: a flat archive ('tar czf up.tar.gz file1
file2') had all its entries dropped and wrote an accepted-but-empty
orig. The layout is now resolved from the leading entries (a lone
top-level directory is held back until the next entry confirms it as
the archive root or proves the archive flat) and flat entries keep
their whole path under the new top-level directory; classic archives
are repacked exactly as before.
2026-09-17 17:45:25 +02:00
vhaudiquet 174a13df39 new: re-probe wizard defaults when packaging a different directory
The detection + probe pass ran against the cwd before the
source-location question, so answering 'another directory' still
offered the cwd's name, version, description, homepage and license
sniff as defaults (only the orig origin followed the chosen tree).
When the answer redirects the wizard to a different directory, the
detection + probe now run again there, feeding every subsequent
probe-derived default; the originally detected directory is not
re-scanned and explicit flags keep winning.
2026-09-17 17:14:25 +02:00
vhaudiquet dd006f7b80 new: validate the command name before generating files
The command/binary name was accepted verbatim and interpolated into
debian/install, debian/rules, debian/tests/smoke, automake variables,
meson.build and [project.scripts]: a value with a space or quote broke
the install lines and shell snippets, 'my.tool' parsed as a nested TOML
table (silently dropping the console script) and produced non-canonical
automake variable names. Both --command and the wizard answer now go
through a shared validator (lowercase identifier: letters, digits,
+ - . _).
2026-09-17 17:03:43 +02:00
vhaudiquet d5b76ec8d8 new: validate wizard defaults like typed answers
ask_text accepted its default on Enter without running the question's
validator, so a probed upstream version like 1.0-2 or v1.0 sailed
through the whole questionnaire and crashed resolve() at the end, and
an invalid git-derived maintainer default (e.g. 'Name <>') was accepted
verbatim. ask_text now takes the validator and applies it to both typed
answers and the offered default — a default that fails validation is
withheld and an invalid answer re-asks — and all question call sites
(incl. the maintainer loop) route through it.
2026-09-17 17:00:22 +02:00
vhaudiquet 70e375a34d new: let an explicit --lang win over build-system detection
The wizard overwrote cli.lang with the detected ecosystem even when the
user passed --lang, and re-asked the language question in the ambiguous
and skeleton cases despite the documented 'flag > detected > default'
merge order. The flag now short-circuits the language step entirely
(detection stays informational); behavior without the flag is
unchanged.
2026-09-17 16:44:35 +02:00
vhaudiquet 8ad50aaf83 put: check the SFTP close status after uploads
ssh2::File's Drop discards the close-handshake error ('too late to
recover'), so a quota or server-side abort surfacing in the final ACKs
was recorded as a successful upload of a truncated file. Close upload
handles explicitly and propagate the error; also applies to the ssh
context driver's write_file and upload_recursive, which had the same
silent-drop issue.
2026-09-17 16:36:36 +02:00
vhaudiquet efb18bfa37 new: fix dead patterns in the generated debian/.gitignore
Patterns containing a slash are anchored relative to the directory
holding the .gitignore, so 'debian/files' inside debian/.gitignore
only ever matched debian/debian/files: every generated pattern was
dead and debhelper artifacts showed up as untracked. Write the
patterns relative to debian/ instead.
2026-09-17 16:33:02 +02:00
vhaudiquet 57db98d776 put: expand ~ in ssh_config IdentityFile paths
IdentityFile values were stored verbatim, so the near-universal
'IdentityFile ~/.ssh/key' spelling never matched an existing file and
the key was silently skipped during authentication. Expand a leading
~ (only that form; ~user and embedded tildes stay verbatim) against
the user's home directory when parsing.
2026-09-17 16:31:04 +02:00
vhaudiquet 775e3d3b8a fmt 2026-09-17 16:27:34 +02:00
vhaudiquet a7cd4244b2 test: hide progress spinners in test runs
Steady-tick spinner threads redraw straight to the real stderr, bypassing
both the harness capture and the per-test log files: 'Scaffolding' lines
from the pkh new tests kept leaking between test results. The scaffold
tests now pass a hidden draw target (the only MultiProgress not created
by the CLI).
2026-09-17 15:32:55 +02:00
vhaudiquet d2bb311f74 net: retry empty index bodies and report by-hash failures in the error
CDNs occasionally answer 200 with a zero-byte body under load; the
checksum verification then reported the empty-string hash as a mismatch,
and the by-hash retry (subject to the same glitch) silently lost its own
failure reason. Treat empty bodies as transient in both fetch paths and
append the by-hash failure to the final VerifyError.

Includes a regression test serving an empty 200 followed by a valid body
on a local socket.
2026-09-17 15:32:55 +02:00
vhaudiquet 4c26122357 test: keep cargo test output quiet with per-test logs and a failure matrix
cargo test used to be unreadable: subprocesses inherited the terminal, so
dpkg-buildpackage, apt and configure output interleaved with the harness
summary, and env_logger lines from parallel tests crossed each other.

New test_support module, compiled into test binaries only (inert stubs
otherwise) and initialized before main via .init_array:

- all log output goes to target/pkh-test-logs/<test>.log, one file per
  test thread, so concurrent tests never interleave
- context-launched commands are captured line by line into the same file
  (driver-level wrapper); test-code spawns use run_logged()
- a panic hook records failures and an atexit callback prints a matrix
  (test name, panic location, message, log path) after the libtest
  summary; tests panicking on purpose can opt out with a guard

Also fixes two test bugs found on the way:

- diff_checkbuilddeps_matrix compared dpkg-checkbuilddeps diagnostics
  against English messages without pinning the locale
- run_source_build in differential tests now captures output like the
  live-UI path does
2026-09-17 15:18:23 +02:00
vhaudiquet 3ed95725e4 net: retry flaky archive fetches and pin index downloads via by-hash
Busy mirrors and CDNs routinely break bulk fetches: pooled keep-alive
connections get closed remotely ('error sending request'), downloads are
cut short (surfacing as bogus checksum mismatches), and index generations
momentarily drift from the Release file fetched moments before.

- shared client: short idle-pool timeout and TCP keepalive, and a
  bounded-retry GET helper now used for index, Release, keyring and
  Launchpad fetches (previously reqwest::get, which has no timeouts)
- downloads: retry the whole download, and check the content length so
  truncation is reported as such instead of a checksum mismatch
- sources index: on a checksum mismatch against the Release file, retry
  pinned to the exact listed generation via Debian's by-hash mechanism;
  body-read errors are retried and reported per component instead of
  aborting the whole lookup
2026-09-17 15:18:14 +02:00
26 changed files with 2950 additions and 339 deletions
+4 -4
View File
@@ -137,7 +137,7 @@ impl ChecksumKind {
}
/// Human-readable algorithm name, for error messages
fn name(self) -> &'static str {
pub fn name(self) -> &'static str {
match self {
ChecksumKind::Md5 => "MD5",
ChecksumKind::Sha1 => "SHA-1",
@@ -782,7 +782,7 @@ pub async fn verify_suite(
/// GET a URL, returning None on any HTTP error, non-success status, or body
/// read failure (Release files are probed, so absence is a normal outcome)
async fn fetch_optional(url: &str) -> Option<Vec<u8>> {
match reqwest::get(url).await {
match crate::distro_info::http_get_retried(url).await {
Ok(response) if response.status().is_success() => match response.bytes().await {
Ok(bytes) => return Some(bytes.to_vec()),
Err(e) => debug!("Reading the body of '{url}' failed: {e}"),
@@ -811,7 +811,7 @@ async fn fetch_keyring_cached(url: &str) -> Result<Vec<u8>, Box<dyn Error + Send
return Ok(cached.clone());
}
let response = reqwest::get(url).await?;
let response = crate::distro_info::http_get_retried(url).await?;
if !response.status().is_success() {
return Err(format!(
"downloading keyring from '{url}' failed with HTTP {}",
@@ -887,7 +887,7 @@ pub async fn ppa_keyring_bytes(
}
let api_url = format!("https://api.launchpad.net/1.0/~{owner}/+archive/ubuntu/{name}");
let response = reqwest::get(&api_url).await?;
let response = crate::distro_info::http_get_retried(&api_url).await?;
if !response.status().is_success() {
return Err(format!(
"querying the Launchpad API for the signing key of PPA \
+181 -13
View File
@@ -75,9 +75,17 @@ pub fn generate_binary_metadata(
let control_content = ctx.read_file(&package_dir.join("debian/control"))?;
let control = ControlInfo::parse_content(&control_content)?;
let files_content = ctx
.read_file(&package_dir.join("debian/files"))
.unwrap_or_default();
// A missing `debian/files` is tolerated (first binary build in a fresh
// tree has nothing registered yet; that surfaces below as the "no binary
// artifacts" error), like `FilesList::load`. Any other read failure must
// not be silently mistaken for an empty registry.
let files_path = package_dir.join("debian/files");
let files_content = if ctx.exists(&files_path)? {
ctx.read_file(&files_path)
.map_err(|e| format!("cannot read '{}': {}", files_path.display(), e))?
} else {
String::new()
};
let mut files_list = FilesList::parse(&files_content)?;
// ------------------------------------------------------------------
@@ -102,6 +110,9 @@ pub fn generate_binary_metadata(
let entry_hashes = hashes
.remove(name)
.ok_or_else(|| format!("artifact '{name}' listed in debian/files but not found"))?;
// SHA-512 stays unknown here: like dpkg-genbuildinfo, no SHA-512
// digest is computed for the artifacts, and an empty digest keeps
// the `Checksums-Sha512` field of the `.buildinfo` omitted.
checksums.insert_entry(
name,
ChecksumEntry {
@@ -109,6 +120,7 @@ pub fn generate_binary_metadata(
md5: entry_hashes.md5,
sha1: entry_hashes.sha1,
sha256: entry_hashes.sha256,
sha512: String::new(),
},
);
// Architecture accumulation in encounter order (dpkg-genchanges).
@@ -130,12 +142,20 @@ pub fn generate_binary_metadata(
let mut source_display = entry.source.clone();
let mut binary_only_changes = None;
if entry.binary_only
&& let Ok(prev_entry) = crate::debian::changelog::parse_previous_version_from_str(
&ctx.read_file(&package_dir.join("debian/changelog"))?,
if entry.binary_only {
// A binary-only upload must reference the previous source version;
// a changelog that cannot yield it is a hard error, like in the
// source-build path. Reuse the changelog read above instead of
// reading the file a second time.
let changelog_path = package_dir.join("debian/changelog");
let prev = crate::debian::changelog::parse_previous_version_from_str(&changelog_content)
.map_err(|e| {
format!(
"cannot parse the previous version from '{}': {e}",
changelog_path.display()
)
&& let Some(prev) = prev_entry
{
})?;
if let Some(prev) = prev {
source_display = format!("{} ({})", entry.source, prev);
binary_only_changes = Some(format!(
"{}\n\n -- {} <{}> {}",
@@ -148,6 +168,7 @@ pub fn generate_binary_metadata(
include_dsc_artifacts(ctx, upload_dir, &dsc_name, &mut checksums)?;
}
}
}
// ------------------------------------------------------------------
// Binary package names and descriptions
@@ -199,9 +220,13 @@ pub fn generate_binary_metadata(
// ------------------------------------------------------------------
// Installed-Build-Depends closure over the context status database
// ------------------------------------------------------------------
// Like the source-build path, a status database that cannot be read is
// a hard error: silently treating it as empty would drop (or gut) the
// `Installed-Build-Depends` field of the produced metadata.
let status_path = Path::new("/var/lib/dpkg/status");
let status_content = ctx
.read_file(Path::new("/var/lib/dpkg/status"))
.unwrap_or_default();
.read_file(status_path)
.map_err(|e| format!("cannot read status file '{}': {}", status_path.display(), e))?;
let bd_fields = [
control.source.get("Build-Depends").unwrap_or(""),
control.source.get("Build-Depends-Arch").unwrap_or(""),
@@ -264,6 +289,9 @@ pub fn generate_binary_metadata(
md5: h.md5.clone(),
sha1: h.sha1.clone(),
sha256: h.sha256.clone(),
// No SHA-512 digest available (see above); keeps the
// `Checksums-Sha512` `.buildinfo` field omitted.
sha512: String::new(),
},
);
}
@@ -311,21 +339,34 @@ fn hashes_in_context(
.map(|n| (n.clone(), ArtifactHashes::default()))
.collect();
// Sizes.
// Sizes. A failed `stat` must fail the metadata generation: an unchecked
// exit status would leave the default size 0 in the produced
// `.changes`/`.buildinfo` checksum entries.
let output = ctx
.command("stat")
.current_dir(dir)
.arg("-c")
.arg("%s %n")
.args(names)
.output()?;
.output()
.map_err(|e| format!("failed to run 'stat' inside the build context: {e}"))?;
if !output.status.success() {
return Err(format!(
"'stat' failed inside the build context: {}",
String::from_utf8_lossy(&output.stderr).trim()
)
.into());
}
let stdout = String::from_utf8_lossy(&output.stdout);
for line in stdout.lines() {
let Some((size, name)) = line.trim().split_once(' ') else {
continue;
};
let size = size
.parse::<u64>()
.map_err(|_| format!("'stat' reported an invalid size '{size}' for '{name}'"))?;
if let Some(slot) = out.get_mut(name) {
slot.size = size.parse().unwrap_or(0);
slot.size = size;
}
}
@@ -423,6 +464,8 @@ fn include_dsc_artifacts(
md5: h.md5.clone(),
sha1: h.sha1.clone(),
sha256: h.sha256.clone(),
// No SHA-512 digest available (see above).
sha512: String::new(),
},
);
}
@@ -440,6 +483,9 @@ fn include_dsc_artifacts(
md5: p.md5.clone().unwrap_or_default(),
sha1: p.sha1.clone().unwrap_or_default(),
sha256: p.sha256.clone().unwrap_or_default(),
// The `.dsc` records no SHA-512 (dpkg only writes
// sha1/sha256 there).
sha512: String::new(),
},
);
}
@@ -580,4 +626,126 @@ Files:
assert_eq!(tar_entry.sha1, "aaa111");
assert_eq!(tar_entry.sha256, "bbb222");
}
/// A binary-only (binNMU) build whose changelog cannot yield the
/// previous version (malformed second header, unbalanced parenthesis)
/// must fail the metadata generation with a diagnostic naming the
/// changelog, instead of silently emitting a plain `Source:` `.changes`
/// with no `Binary-Only-Changes` and no redistributed previous `.dsc`.
#[test]
fn binary_only_prev_version_parse_failure_errors_instead_of_wrong_metadata() {
let changelog = "\
hello (1.0-1+b1) unstable; urgency=medium, binary-only=yes
* Binary-only rebuild.
-- A B <a@b.c> Mon, 01 Jan 2024 00:00:00 +0000
hello (1.0-1 unstable; urgency=medium
* Previous entry with an unbalanced parenthesis.
-- A B <a@b.c> Sun, 31 Dec 2023 00:00:00 +0000
";
let control = "\
Source: hello
Section: devel
Priority: optional
Maintainer: A B <a@b.c>
Package: hello
Architecture: all
Description: test package
";
let base = tempfile::tempdir().expect("tempdir");
let tree = base.path().join("hello-1.0");
std::fs::create_dir_all(tree.join("debian")).expect("mkdir tree");
std::fs::write(tree.join("debian/changelog"), changelog).expect("write changelog");
std::fs::write(tree.join("debian/control"), control).expect("write control");
std::fs::write(
tree.join("debian/files"),
"hello_1.0-1+b1_all.deb devel optional\n",
)
.expect("write files");
std::fs::write(base.path().join("hello_1.0-1+b1_all.deb"), "deb payload")
.expect("write deb");
let ctx = Arc::new(
crate::context::Context::new(crate::context::ContextConfig::Local).expect("context"),
);
let opts = BinaryMetadataOptions {
profiles: Vec::new(),
vendor: "debian".to_string(),
exported_env: BTreeMap::new(),
build_arch: "amd64".to_string(),
host_arch: "amd64".to_string(),
};
let err = generate_binary_metadata(&ctx, &tree, base.path(), &opts)
.expect_err("binary-only build with an unparseable changelog must fail");
let err = err.to_string();
assert!(err.contains("debian/changelog"), "{err}");
assert!(err.contains("previous version"), "{err}");
assert!(err.contains("unbalanced parenthesis"), "{err}");
}
/// An unreadable `debian/files` (e.g. permissions) must fail the
/// metadata generation with an error naming the read failure, instead of
/// being silently treated as an empty registry and reported as "no
/// binary artifacts found". A *missing* file stays tolerated (first
/// build in a fresh tree); the distinction matters.
#[test]
fn unreadable_debian_files_errors_instead_of_empty_registry() {
if crate::utils::root::is_root().unwrap_or(false) {
// Root can read files regardless of permissions.
return;
}
let changelog = "\
hello (1.0-1) unstable; urgency=medium
* Regular build.
-- A B <a@b.c> Mon, 01 Jan 2024 00:00:00 +0000
";
let control = "\
Source: hello
Section: devel
Priority: optional
Maintainer: A B <a@b.c>
Package: hello
Architecture: all
Description: test package
";
let base = tempfile::tempdir().expect("tempdir");
let tree = base.path().join("hello-1.0");
std::fs::create_dir_all(tree.join("debian")).expect("mkdir tree");
std::fs::write(tree.join("debian/changelog"), changelog).expect("write changelog");
std::fs::write(tree.join("debian/control"), control).expect("write control");
let files_path = tree.join("debian/files");
std::fs::write(&files_path, "hello_1.0-1_all.deb devel optional\n").expect("write files");
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&files_path, std::fs::Permissions::from_mode(0o000))
.expect("chmod files");
}
let ctx = Arc::new(
crate::context::Context::new(crate::context::ContextConfig::Local).expect("context"),
);
let opts = BinaryMetadataOptions {
profiles: Vec::new(),
vendor: "debian".to_string(),
exported_env: BTreeMap::new(),
build_arch: "amd64".to_string(),
host_arch: "amd64".to_string(),
};
let err = generate_binary_metadata(&ctx, &tree, base.path(), &opts)
.expect_err("unreadable debian/files must fail with a read error");
let err = err.to_string();
assert!(err.contains("cannot read"), "{err}");
assert!(err.contains("debian/files"), "{err}");
#[cfg(unix)]
assert!(err.contains("Permission denied"), "{err}");
}
}
+149
View File
@@ -207,6 +207,13 @@ pub fn installed_build_depends_from_content(
entries.sort_by(|a, b| a.0.cmp(&b.0).then_with(|| a.1.cmp(&b.1)));
entries.dedup_by(|a, b| a.0 == b.0);
// With no reachable entries, return an empty value so `render_buildinfo`
// omits the field entirely; a leading `\n` alone would render a
// malformed `Installed-Build-Depends:` with only a blank continuation.
if entries.is_empty() {
return Ok(String::new());
}
// Leading `\n`: pre-wrapped multiline field, dpkg-style.
let mut out = String::from("\n");
out.push_str(
@@ -289,6 +296,12 @@ pub fn render_buildinfo(input: &BuildInfoInput) -> Paragraph {
p.set("Checksums-Md5", &input.checksums.field_md5());
p.set("Checksums-Sha1", &input.checksums.field_sha1());
p.set("Checksums-Sha256", &input.checksums.field_sha256());
// Only-if-populated: entries merged from a `.dsc` carry no SHA-512
// (dpkg only records sha1/sha256 there), and an incomplete checksum
// list must never be rendered.
if let Some(sha512) = input.checksums.field_sha512() {
p.set("Checksums-Sha512", &sha512);
}
}
p.set("Build-Origin", &input.build_origin);
p.set("Build-Architecture", &input.build_architecture);
@@ -378,6 +391,68 @@ Architecture: amd64
assert!(!ibd.contains("not-installed"));
}
/// With no installed entries reachable (empty status database), the
/// computed value must be EMPTY so `render_buildinfo` omits the
/// `Installed-Build-Depends` field entirely, instead of emitting a
/// malformed field with only a blank continuation line.
#[test]
fn installed_build_depends_without_entries_is_empty_and_omitted() {
let ibd = installed_build_depends_from_content("", &["libc6"]).unwrap();
assert_eq!(
ibd, "",
"zero entries must yield an empty value, not \"\\n\""
);
let input = BuildInfoInput {
source: "hello".to_string(),
binaries: vec!["hello".to_string()],
architecture: "amd64".to_string(),
version: "1.0".to_string(),
binary_only_changes: None,
build_origin: "debian".to_string(),
build_architecture: "amd64".to_string(),
build_date: "Sat, 22 Aug 2026 23:08:42 +0200".to_string(),
checksums: FileChecksums::new(),
installed_build_depends: ibd,
environment: String::new(),
};
let p = render_buildinfo(&input);
assert!(
p.get("Installed-Build-Depends").is_none(),
"empty value must omit the field entirely"
);
}
/// With installed entries, the value keeps the dpkg-style leading `\n`
/// (pre-wrapped multiline field) and the field is rendered.
#[test]
fn installed_build_depends_with_entries_renders_field() {
let status = "\
Package: gcc
Status: install ok installed
Version: 13.2
Architecture: amd64
";
let ibd = installed_build_depends_from_content(status, &["gcc"]).unwrap();
assert_eq!(ibd, "\ngcc (= 13.2)");
let input = BuildInfoInput {
source: "hello".to_string(),
binaries: vec!["hello".to_string()],
architecture: "amd64".to_string(),
version: "1.0".to_string(),
binary_only_changes: None,
build_origin: "debian".to_string(),
build_architecture: "amd64".to_string(),
build_date: "Sat, 22 Aug 2026 23:08:42 +0200".to_string(),
checksums: FileChecksums::new(),
installed_build_depends: ibd,
environment: String::new(),
};
let p = render_buildinfo(&input);
assert_eq!(p.get("Installed-Build-Depends"), Some("\ngcc (= 13.2)"));
}
#[test]
fn wrap_binary_field() {
assert_eq!(wrap_long("abc"), "abc");
@@ -425,4 +500,78 @@ Architecture: amd64
);
assert_eq!(p.get("Format"), Some("1.0"));
}
/// `Checksums-Sha512` is emitted (after `Checksums-Sha256`) only when
/// every distributed file has a SHA-512 digest; entries merged without
/// one (e.g. taken from a `.dsc`) omit the field entirely instead of
/// rendering an incomplete checksum list.
#[test]
fn checksums_sha512_emitted_only_when_populated() {
let dir = tempfile::tempdir().unwrap();
let artifact = dir.path().join("hello_1.0_all.deb");
std::fs::write(&artifact, b"deb payload").unwrap();
let mut checksums = FileChecksums::new();
checksums.add_file(&artifact).unwrap();
let mk_input = |checksums: FileChecksums| BuildInfoInput {
source: "hello".to_string(),
binaries: vec![],
architecture: "all".to_string(),
version: "1.0".to_string(),
binary_only_changes: None,
build_origin: "debian".to_string(),
build_architecture: "amd64".to_string(),
build_date: "Sat, 22 Aug 2026 23:08:42 +0200".to_string(),
checksums,
installed_build_depends: String::new(),
environment: String::new(),
};
// All digests computed: the field is present and parses back.
let p = render_buildinfo(&mk_input(checksums.clone()));
let keys: Vec<&str> = p.iter().map(|(k, _)| k).collect();
assert_eq!(
keys,
vec![
"Format",
"Source",
"Architecture",
"Version",
"Checksums-Md5",
"Checksums-Sha1",
"Checksums-Sha256",
"Checksums-Sha512",
"Build-Origin",
"Build-Architecture",
"Build-Date",
]
);
let sha512_field = p.get("Checksums-Sha512").unwrap();
let parsed =
FileChecksums::parse_field(crate::debian::ChecksumKind::Sha512, sha512_field).unwrap();
assert_eq!(parsed.len(), 1);
assert_eq!(parsed[0].0, "hello_1.0_all.deb");
assert_eq!(
parsed[0].1.sha512,
checksums.get("hello_1.0_all.deb").unwrap().sha512
);
// An entry without SHA-512 (as merged from a `.dsc`) suppresses the
// field; the other Checksums fields keep listing every file.
checksums.insert_entry(
"hello_1.0.orig.tar.xz",
crate::debian::ChecksumEntry {
size: 3,
md5: checksums.get("hello_1.0_all.deb").unwrap().md5.clone(),
sha1: String::new(),
sha256: String::new(),
sha512: String::new(),
},
);
let p = render_buildinfo(&mk_input(checksums));
assert!(p.get("Checksums-Sha512").is_none());
let sha256_lines = p.get("Checksums-Sha256").unwrap().lines();
assert_eq!(sha256_lines.filter(|l| !l.is_empty()).count(), 2);
}
}
+120 -7
View File
@@ -3,7 +3,7 @@
//! sanitized environment recorded in `.buildinfo` files.
use std::collections::BTreeMap;
use std::path::Path;
use std::path::{Path, PathBuf};
/// Number of parallel jobs to advertise in `DEB_BUILD_OPTIONS`.
pub fn num_parallel() -> usize {
@@ -12,12 +12,36 @@ pub fn num_parallel() -> usize {
.unwrap_or(1)
}
/// Merge an inherited `DEB_BUILD_OPTIONS` value with options pkh computes
/// itself.
///
/// `dpkg-buildpackage` prepends the environment's `DEB_BUILD_OPTIONS` to the
/// options it derives (`parallel=N`, ...), so caller-set options such as
/// `terse` or `nocheck` survive alongside pkh's own. The result is therefore
/// the inherited options followed by `computed`, space-separated; each side is
/// trimmed and its internal whitespace runs collapsed. An unset or blank
/// inherited value yields just `computed`.
pub fn merge_deb_build_options(inherited: Option<&str>, computed: &str) -> String {
let computed = normalize_build_options(computed);
match inherited.map(normalize_build_options) {
Some(inherited) if !inherited.is_empty() => format!("{} {}", inherited, computed),
_ => computed,
}
}
/// Trim and collapse internal whitespace in a `DEB_BUILD_OPTIONS` fragment.
fn normalize_build_options(options: &str) -> String {
options.split_whitespace().collect::<Vec<_>>().join(" ")
}
/// Compute the environment variables exported before running any build step.
///
/// Mirrors dpkg behavior:
/// - `SOURCE_DATE_EPOCH` from the changelog entry timestamp
/// (<https://reproducible-builds.org/specs/source-date-epoch/>),
/// - `DEB_BUILD_OPTIONS=parallel=N` (auto-detected job count),
/// - `DEB_BUILD_OPTIONS`: any value inherited from the invoking environment
/// (dpkg-buildpackage prepends it) followed by `parallel=N` (auto-detected
/// job count),
/// - `DEB_BUILD_PROFILES` when non-default profiles are requested.
///
/// The locale is pinned to `C` (`LC_ALL`, which takes precedence over any
@@ -38,7 +62,10 @@ pub fn build_env(
);
env.insert(
"DEB_BUILD_OPTIONS".to_string(),
format!("parallel={}", parallel),
merge_deb_build_options(
std::env::var("DEB_BUILD_OPTIONS").ok().as_deref(),
&format!("parallel={}", parallel),
),
);
if !build_profiles.is_empty() {
env.insert("DEB_BUILD_PROFILES".to_string(), build_profiles.join(","));
@@ -57,15 +84,31 @@ pub fn arch_env(host_arch: Option<&str>) -> Result<BTreeMap<String, String>, Str
crate::debian::arch::arch_env(host_arch)
}
/// Read the current vendor name from `/etc/dpkg/origins/default`
/// (its `Vendor:` or `Origin:` field), defaulting to `"debian"`.
/// Read the current vendor name from the active dpkg origins `default` file
/// (`$DPKG_ORIGINS_DIR/default`, falling back to `/etc/dpkg/origins/default`;
/// its `Vendor:` or `Origin:` field), defaulting to `"debian"`.
pub fn current_vendor() -> String {
std::fs::read_to_string(Path::new("/etc/dpkg/origins/default"))
let path = resolve_origins_default(
std::env::var("DPKG_ORIGINS_DIR").ok().as_deref(),
"/etc/dpkg/origins",
);
std::fs::read_to_string(path)
.ok()
.and_then(|content| vendor_from_origins_content(&content))
.unwrap_or_else(|| "debian".to_string())
}
/// Resolve the path of the active dpkg origins file from the
/// `DPKG_ORIGINS_DIR` value (the directory holding the origin files, where
/// `default` selects the active one) and the fallback directory
/// (`/etc/dpkg/origins`). An unset or empty directory value falls back.
fn resolve_origins_default(origins_dir: Option<&str>, fallback_dir: &str) -> PathBuf {
let dir = origins_dir
.filter(|d| !d.is_empty())
.unwrap_or(fallback_dir);
Path::new(dir).join("default")
}
/// Extract the vendor name from the content of a dpkg origins file: its
/// `Vendor:` field, falling back to `Origin:` when absent. `None` when
/// neither field carries a non-empty value.
@@ -271,13 +314,63 @@ mod tests {
assert_eq!(env.get("LANG").unwrap(), "C");
assert_eq!(env.get("LC_ALL").unwrap(), "C");
assert_eq!(env.get("SOURCE_DATE_EPOCH").unwrap(), "1787392800");
assert_eq!(env.get("DEB_BUILD_OPTIONS").unwrap(), "parallel=16");
// Reading the var is race-free; the expected value goes through the
// same merge so the assertion holds whatever the ambient environment
// carries.
let expected = merge_deb_build_options(
std::env::var("DEB_BUILD_OPTIONS").ok().as_deref(),
"parallel=16",
);
assert_eq!(env.get("DEB_BUILD_OPTIONS").unwrap(), &expected);
assert!(!env.contains_key("DEB_BUILD_PROFILES"));
let env = build_env(1, 4, &["nodoc".to_string(), "cross".to_string()]);
assert_eq!(env.get("DEB_BUILD_PROFILES").unwrap(), "nodoc,cross");
}
/// dpkg-buildpackage prepends the inherited `DEB_BUILD_OPTIONS`, so
/// user-set options survive alongside the computed ones.
#[test]
fn merge_prepends_inherited_options() {
assert_eq!(
merge_deb_build_options(Some("terse"), "parallel=16"),
"terse parallel=16"
);
assert_eq!(
merge_deb_build_options(Some("nocheck terse"), "parallel=4"),
"nocheck terse parallel=4"
);
}
/// An unset, empty or blank inherited value yields just the computed
/// options.
#[test]
fn merge_skips_empty_inherited() {
assert_eq!(merge_deb_build_options(None, "parallel=8"), "parallel=8");
assert_eq!(
merge_deb_build_options(Some(""), "parallel=8"),
"parallel=8"
);
assert_eq!(
merge_deb_build_options(Some(" "), "parallel=8"),
"parallel=8"
);
}
/// Both sides are trimmed and internal whitespace runs collapsed: no
/// leading/trailing space, no double spaces in the merged result.
#[test]
fn merge_normalizes_whitespace() {
assert_eq!(
merge_deb_build_options(Some(" terse "), "parallel=2"),
"terse parallel=2"
);
assert_eq!(
merge_deb_build_options(Some("nocheck\t terse"), "parallel=2"),
"nocheck terse parallel=2"
);
}
#[test]
fn vendor_defaults() {
assert!(default_build_profiles("debian").is_empty());
@@ -306,6 +399,26 @@ mod tests {
assert_eq!(vendor_from_origins_content("Suite: stable\n"), None);
}
/// `current_vendor` must honor `DPKG_ORIGINS_DIR` (already on the
/// `.buildinfo` allow-list) when locating the `default` origins file,
/// falling back to `/etc/dpkg/origins/default` when unset or empty.
#[test]
fn origins_default_path_honors_dpkg_origins_dir() {
assert_eq!(
resolve_origins_default(Some("/custom/origins"), "/etc/dpkg/origins"),
PathBuf::from("/custom/origins/default")
);
assert_eq!(
resolve_origins_default(None, "/etc/dpkg/origins"),
PathBuf::from("/etc/dpkg/origins/default")
);
// An empty value behaves as unset, like dpkg's `$dir || $default`.
assert_eq!(
resolve_origins_default(Some(""), "/etc/dpkg/origins"),
PathBuf::from("/etc/dpkg/origins/default")
);
}
#[test]
fn environment_escaping() {
// The function reads the process env; just verify formatting helpers
+89 -24
View File
@@ -72,11 +72,19 @@ pub fn build_source_package(
cwd: Option<&Path>,
ui: Option<Arc<DebUi>>,
) -> Result<(), Box<dyn Error>> {
let cwd = cwd.unwrap_or_else(|| Path::new("."));
let output = match run_source_build(cwd, &SourceBuildOptions::default(), ui.clone()) {
// Default to the process's current working directory, resolved to an
// absolute path: the output directory is derived from `cwd.parent()`
// downstream, which only yields a real directory for an absolute `cwd`
// (the parent of "." is the empty path).
let cwd = match cwd {
Some(p) => p.to_path_buf(),
None => std::env::current_dir()
.map_err(|e| format!("cannot determine the current working directory: {e}"))?,
};
let output = match run_source_build(&cwd, &SourceBuildOptions::default(), ui.clone()) {
Ok(output) => output,
Err(e) if e.downcast_ref::<VendorDriftError>().is_some() => {
return retry_after_revendor(cwd, ui, e);
return retry_after_revendor(&cwd, ui, e);
}
Err(e) => {
if let Some(u) = &ui {
@@ -210,7 +218,12 @@ pub fn run_source_build(
opts: &SourceBuildOptions,
ui: Option<Arc<DebUi>>,
) -> Result<SourceBuildOutput, Box<dyn Error>> {
let sink: Option<Arc<dyn LineSink>> = ui.as_ref().map(|u| u.sink());
// Without a live UI, test runs still capture command output into the
// per-test log file instead of letting it inherit the terminal
let sink: Option<Arc<dyn LineSink>> = ui
.as_ref()
.map(|u| u.sink())
.or_else(crate::test_support::subprocess_sink);
// ------------------------------------------------------------------
// 1. Sanity checks
// ------------------------------------------------------------------
@@ -349,6 +362,10 @@ pub fn run_source_build(
.get("DEB_HOST_ARCH")
.cloned()
.unwrap_or_else(|| crate::debian::arch::native().unwrap_or_default()),
build_arch: arch_vars
.get("DEB_BUILD_ARCH")
.cloned()
.unwrap_or_else(|| crate::debian::arch::native().unwrap_or_default()),
build_profiles: profiles.clone(),
..Default::default()
};
@@ -527,6 +544,10 @@ pub fn run_source_build(
md5: partial.md5.clone().unwrap_or_default(),
sha1: partial.sha1.clone().unwrap_or_default(),
sha256: partial.sha256.clone().unwrap_or_default(),
// The .dsc records no SHA-512 (dpkg only writes sha1/sha256
// there); the empty digest keeps the .buildinfo's
// `Checksums-Sha512` field omitted.
sha512: String::new(),
},
);
tarball_paths.push(path);
@@ -798,7 +819,9 @@ struct CommandFailure {
/// it (live view + tee log) while the stderr is additionally captured for
/// the failure classification; otherwise stdio is inherited from the
/// terminal. Returns an error (with the captured stderr) on non-zero exit
/// status.
/// status. A panicking stdout/stderr reader thread also yields an error
/// (the captured output would be incomplete), but only after the command's
/// own failure, which takes precedence.
fn run_command_capturing(
cwd: &Path,
program: &str,
@@ -820,6 +843,8 @@ fn run_command_capturing(
// classification.
let stderr_capture = Arc::new(std::sync::Mutex::new(String::new()));
// Printable panic message from a reader thread, if one died mid-pump.
let mut reader_panic = None;
let status = match sink {
None => cmd.status().map_err(|e| CommandFailure {
error: format!("failed to run '{}': {}", program, e).into(),
@@ -856,8 +881,11 @@ fn run_command_capturing(
);
}
});
let _ = out_thread.join();
let _ = err_thread.join();
// The threads end on EOF, i.e. once the child exited and closed
// its streams; a panic from either means the captured output is
// incomplete.
reader_panic = reader_panic_message(out_thread.join())
.or_else(|| reader_panic_message(err_thread.join()));
child.wait().map_err(|e| CommandFailure {
error: format!("failed to wait for '{}': {}", program, e).into(),
@@ -867,6 +895,11 @@ fn run_command_capturing(
};
if !status.success() {
// The command's own failure takes precedence over a dead reader; the
// panic is still logged so the truncated-log cause is not lost.
if let Some(message) = &reader_panic {
log::error!("the build output reader failed: {message}");
}
return Err(CommandFailure {
error: format!(
"'{} {}' failed with status: {}",
@@ -878,9 +911,30 @@ fn run_command_capturing(
stderr: std::mem::take(&mut stderr_capture.lock().unwrap_or_else(|e| e.into_inner())),
});
}
// The command succeeded but a reader thread panicked: the captured output
// (live view + tee log) is incomplete, so this cannot pass as a success.
if let Some(message) = reader_panic {
return Err(CommandFailure {
error: format!("the build output reader failed: {message}").into(),
stderr: std::mem::take(&mut stderr_capture.lock().unwrap_or_else(|e| e.into_inner())),
});
}
Ok(())
}
/// Extract a printable message from a reader-thread join result; `None` when
/// the thread finished normally.
fn reader_panic_message(join: std::thread::Result<()>) -> Option<String> {
join.err().map(|payload| {
payload
.downcast_ref::<&str>()
.map(|s| (*s).to_string())
.or_else(|| payload.downcast_ref::<String>().cloned())
.unwrap_or_else(|| "non-string panic payload".to_string())
})
}
/// Run a build command, discarding the captured stderr.
fn run_command(
cwd: &Path,
@@ -1304,7 +1358,7 @@ mod differential_tests {
}
}
cmd.arg("-f").arg(&tarball).arg(&dir_name);
let status = cmd.status().expect("run tar");
let status = crate::test_support::run_logged(&mut cmd).expect("run tar");
assert!(status.success(), "tar failed for {}", tarball.display());
}
@@ -1312,11 +1366,8 @@ mod differential_tests {
}
fn copy_path(src: &Path, dst_root: &Path) {
let status = Command::new("cp")
.arg("-a")
.arg(src)
.arg(dst_root)
.status()
let status =
crate::test_support::run_logged(Command::new("cp").arg("-a").arg(src).arg(dst_root))
.expect("run cp -a");
assert!(
status.success(),
@@ -1327,10 +1378,16 @@ mod differential_tests {
}
fn run_dpkg(tree: &Path) {
let status = Command::new("dpkg-buildpackage")
.current_dir(tree)
.args(["-S", "-I", "-i", "-nc", "-d", "--no-sign"])
.status()
let status = crate::test_support::run_logged(
Command::new("dpkg-buildpackage").current_dir(tree).args([
"-S",
"-I",
"-i",
"-nc",
"-d",
"--no-sign",
]),
)
.expect("failed to run dpkg-buildpackage (is dpkg-dev installed?)");
assert!(status.success(), "dpkg-buildpackage failed");
}
@@ -1551,8 +1608,12 @@ mod differential_tests {
// builtin dependencies (build-essential:native), matching the
// native checker which knows no builtins. All options must precede
// the control-file operand (POSIX-style option parsing).
// The diagnostics are compared against the native checker's English
// messages, so the tool must run under the C locale regardless of
// the host configuration.
let output = Command::new("dpkg-checkbuilddeps")
.current_dir(dir.path())
.env("LC_ALL", "C")
.arg("--admindir")
.arg(&admindir)
.args(args)
@@ -1587,8 +1648,10 @@ mod differential_tests {
}
i += 1;
}
let host_arch = crate::debian::arch::native().unwrap_or_else(|_| "amd64".into());
let opts = crate::debian::deps::CheckOpts {
host_arch: crate::debian::arch::native().unwrap_or_else(|_| "amd64".into()),
host_arch: host_arch.clone(),
build_arch: host_arch,
build_profiles: profiles,
ignore_arch,
ignore_indep,
@@ -1782,10 +1845,11 @@ Provides: virtual-thing (= 2.0), plain-virtual
let ours_tree = write_tree(&ours_root);
// Golden side: real dpkg-buildpackage binary build.
let status = Command::new("dpkg-buildpackage")
let status = crate::test_support::run_logged(
Command::new("dpkg-buildpackage")
.current_dir(&golden_tree)
.args(["-b", "-d", "--no-sign"])
.status()
.args(["-b", "-d", "--no-sign"]),
)
.expect("run dpkg-buildpackage (is dpkg-dev installed?)");
assert!(status.success(), "golden dpkg-buildpackage -b failed");
@@ -1811,11 +1875,12 @@ Provides: virtual-thing (= 2.0), plain-virtual
.collect();
for target in ["build", "binary"] {
let status = Command::new("debian/rules")
let status = crate::test_support::run_logged(
Command::new("debian/rules")
.current_dir(&ours_tree)
.envs(build_env_vars.clone())
.arg(target)
.status()
.arg(target),
)
.expect("run rules target");
assert!(status.success(), "debian/rules {target} failed");
}
+3 -1
View File
@@ -311,7 +311,9 @@ impl Context {
overlay_mounts: std::sync::Mutex::new(Vec::new()),
}),
};
*driver_lock = Some(driver);
// In test runs, commands whose output would inherit the terminal
// are captured into the per-test log file instead
*driver_lock = Some(crate::test_support::wrap_driver(driver));
}
driver_lock
}
+4
View File
@@ -8,6 +8,10 @@ mod ssh;
mod unshare;
pub use api::{Context, ContextCommand, ContextConfig, LineSink, Stream};
// The driver trait is implementation detail of the context API; it is only
// needed crate-internally (test-run capture wrapper), so keep it out of the
// public surface (and its documentation requirement).
pub(crate) use api::ContextDriver;
pub use manager::ContextManager;
use std::sync::Arc;
+19
View File
@@ -288,6 +288,13 @@ impl ContextDriver for SshDriver {
}
let mut remote_file = sftp.create(path).map_err(io::Error::other)?;
remote_file.write_all(content.as_bytes())?;
// Close explicitly: the `Drop` impl of `ssh2::File` discards a
// close-time error ("too late to recover"), silently truncating the
// remote file. Writes are unbuffered (`Write::flush` is a no-op), so
// no flush is needed before closing.
remote_file.close().map_err(|e| {
io::Error::other(format!("Failed to close remote file {:?}: {}", path, e))
})?;
Ok(())
}
@@ -327,6 +334,18 @@ impl SshDriver {
io::Error::other(format!("Failed to create remote file {:?}: {}", dest, e))
})?;
io::copy(&mut file, &mut remote_file)?;
// Close explicitly: quota-exceeded and similar failures only
// surface in the final ACKs and the close handshake, and the
// `Drop` impl of `ssh2::File` discards that error ("too late to
// recover"), leaving a truncated remote file behind. Writes are
// unbuffered (`ssh2::File`'s `Write::flush` is a no-op), so no
// flush is needed before closing.
remote_file.close().map_err(|e| {
io::Error::other(format!(
"Failed to close remote file {:?} after upload: {}",
dest, e
))
})?;
}
Ok(())
}
+3
View File
@@ -887,6 +887,9 @@ mod cleanup_registry_tests {
#[test]
fn panicking_hook_does_not_skip_the_others() {
let _serial = test_lock();
// The hook below panics on purpose: do not record it as a test
// failure in the end-of-run matrix
let _quiet = crate::test_support::suppress_failure_recording();
let (before, ran_before) = counting_hook();
let boom = register_cleanup_hook(Box::new(|| panic!("cleanup exploded")));
+267 -3
View File
@@ -1,5 +1,5 @@
//! File checksum computation and formatting for `.changes` / `.buildinfo`
//! fields (MD5, SHA-1, SHA-256 + size), mirroring `Dpkg::Checksums`.
//! fields (MD5, SHA-1, SHA-256, SHA-512 + size), mirroring `Dpkg::Checksums`.
use std::collections::HashMap;
use std::io::Read;
@@ -7,7 +7,7 @@ use std::path::Path;
use md5::Md5;
use sha1::Sha1;
use sha2::{Digest, Sha256};
use sha2::{Digest, Sha256, Sha512};
/// Checksums and size of a single file.
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -20,6 +20,40 @@ pub struct Entry {
pub sha1: String,
/// Lowercase hexadecimal SHA-256 digest.
pub sha256: String,
/// Lowercase hexadecimal SHA-512 digest.
pub sha512: String,
}
/// The checksum algorithm carried by a `Checksums-*` field body, as handled
/// by [`FileChecksums::parse_field`].
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ChecksumKind {
/// SHA-1 (`Checksums-Sha1` field).
Sha1,
/// SHA-256 (`Checksums-Sha256` field).
Sha256,
/// SHA-512 (`Checksums-Sha512` field).
Sha512,
}
impl ChecksumKind {
/// The `Checksums-*` field name carrying this digest.
pub fn field_name(self) -> &'static str {
match self {
ChecksumKind::Sha1 => "Checksums-Sha1",
ChecksumKind::Sha256 => "Checksums-Sha256",
ChecksumKind::Sha512 => "Checksums-Sha512",
}
}
/// Length in lowercase hex characters of one digest of this kind.
fn digest_len(self) -> usize {
match self {
ChecksumKind::Sha1 => 40,
ChecksumKind::Sha256 => 64,
ChecksumKind::Sha512 => 128,
}
}
}
/// Compute all supported checksums of a file.
@@ -30,6 +64,7 @@ fn compute(path: &Path) -> Result<Entry, Box<dyn std::error::Error>> {
let mut md5_hasher = Md5::new();
let mut sha1_hasher = Sha1::new();
let mut sha256_hasher = Sha256::new();
let mut sha512_hasher = Sha512::new();
let mut size: u64 = 0;
let mut buf = [0u8; 64 * 1024];
@@ -41,6 +76,7 @@ fn compute(path: &Path) -> Result<Entry, Box<dyn std::error::Error>> {
md5_hasher.update(&buf[..n]);
sha1_hasher.update(&buf[..n]);
sha256_hasher.update(&buf[..n]);
sha512_hasher.update(&buf[..n]);
size += n as u64;
}
@@ -49,6 +85,7 @@ fn compute(path: &Path) -> Result<Entry, Box<dyn std::error::Error>> {
md5: hex::encode(md5_hasher.finalize()),
sha1: hex::encode(sha1_hasher.finalize()),
sha256: hex::encode(sha256_hasher.finalize()),
sha512: hex::encode(sha512_hasher.finalize()),
})
}
@@ -168,6 +205,84 @@ impl FileChecksums {
pub fn field_sha256(&self) -> String {
self.format_field(|e| &e.sha256)
}
/// Value for the `Checksums-Sha512` field, or `None` when any registered
/// file has no SHA-512 digest (e.g. entries merged from a `.dsc`, which
/// dpkg only writes with sha1/sha256 checksums): renderers omit the
/// field instead of writing an incomplete checksum list.
pub fn field_sha512(&self) -> Option<String> {
if self.iter().any(|(_, e)| e.sha512.is_empty()) {
return None;
}
Some(self.format_field(|e| &e.sha512))
}
/// Parse the body of a `Checksums-Sha1` / `Checksums-Sha256` /
/// `Checksums-Sha512` field (as rendered by [`FileChecksums::field_sha1`],
/// [`FileChecksums::field_sha256`] or [`FileChecksums::field_sha512`])
/// into `(name, entry)` pairs, ready to be fed into
/// [`FileChecksums::insert_entry`] (e.g. when consuming a `.dsc`).
///
/// Each non-blank line holds `"<hex digest> <size> <name>"`; blank lines
/// are tolerated and anything else is a malformed line, reported as an
/// error naming [`ChecksumKind::field_name`] and the offending line. Only
/// the digest selected by `kind` is filled in the returned entries: the
/// other digest fields are left empty and must be completed from the
/// remaining `Checksums-*` fields (or by recomputation) before rendering.
///
/// Note: this deliberately re-implements the line grammar of the private
/// `build::parse_checksum_field` helper (which additionally accepts the
/// legacy 5-column `Files` layout); the two are intentionally not unified
/// across modules.
pub fn parse_field(kind: ChecksumKind, value: &str) -> Result<Vec<(String, Entry)>, String> {
let mut entries = Vec::new();
for line in value.lines() {
if line.trim().is_empty() {
continue;
}
let tokens: Vec<&str> = line.split_whitespace().collect();
let [digest, size, name] = tokens.as_slice() else {
return Err(format!(
"malformed '{}' line (expected 'checksum size name', got {} \
columns): '{line}'",
kind.field_name(),
tokens.len()
));
};
let size: u64 = size.parse().map_err(|_| {
format!(
"malformed '{}' line (size '{size}' is not a number): '{line}'",
kind.field_name()
)
})?;
let digest_ok = digest.len() == kind.digest_len()
&& digest
.bytes()
.all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b));
if !digest_ok {
return Err(format!(
"malformed '{}' line (digest '{digest}' is not {} lowercase \
hex characters): '{line}'",
kind.field_name(),
kind.digest_len()
));
}
let mut entry = Entry {
size,
md5: String::new(),
sha1: String::new(),
sha256: String::new(),
sha512: String::new(),
};
match kind {
ChecksumKind::Sha1 => entry.sha1 = digest.to_string(),
ChecksumKind::Sha256 => entry.sha256 = digest.to_string(),
ChecksumKind::Sha512 => entry.sha512 = digest.to_string(),
}
entries.push((name.to_string(), entry));
}
Ok(entries)
}
}
#[cfg(test)]
@@ -184,16 +299,165 @@ mod tests {
cs.add_file(&p).unwrap();
let e = cs.get("sample.txt").unwrap();
// Verified with coreutils: echo "hello world" | md5sum / sha1sum / sha256sum
// Verified with coreutils: echo "hello world" | md5sum / sha1sum / sha256sum / sha512sum
assert_eq!(e.md5, "6f5902ac237024bdd0c176cb93063dc4");
assert_eq!(e.sha1, "22596363b3de40b06f981fb85d82312e8c0ed511");
assert_eq!(
e.sha256,
"a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447"
);
assert_eq!(
e.sha512,
"db3974a97f2407b7cae1ae637c0030687a11913274d578492558e39c16c017de\
84eacdc8c62fe34ee4e12b4b1428817f09b6a2760c3f8a664ceae94d2434a593"
);
assert_eq!(e.size, 12);
}
/// SHA-512 of the empty input is a well-known constant: a zero-size file
/// must still carry it (never an empty digest string, which is reserved
/// for "digest unknown", e.g. entries merged from a `.dsc`).
#[test]
fn sha512_of_empty_file_is_the_known_constant() {
let dir = tempfile::tempdir().unwrap();
let p = dir.path().join("empty.txt");
std::fs::write(&p, b"").unwrap();
let mut cs = FileChecksums::new();
cs.add_file(&p).unwrap();
let e = cs.get("empty.txt").unwrap();
assert_eq!(e.size, 0);
assert_eq!(
e.sha512,
"cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce\
47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e"
);
}
/// All four digests render; the `Checksums-Sha512` field round-trips
/// through [`FileChecksums::parse_field`] back into a registry with
/// identical names (insertion order), sizes and SHA-512 digests.
#[test]
fn sha512_field_round_trip() {
let dir = tempfile::tempdir().unwrap();
let a = dir.path().join("a.txt");
let b = dir.path().join("b.txt");
std::fs::write(&a, b"aaa").unwrap();
std::fs::write(&b, b"bb").unwrap();
let mut cs = FileChecksums::new();
cs.add_file(&b).unwrap();
cs.add_file(&a).unwrap();
// Every digest kind must be populated and render a full field.
assert!(!cs.field_md5().is_empty());
assert!(!cs.field_sha1().is_empty());
assert!(!cs.field_sha256().is_empty());
let sha512_field = cs.field_sha512().expect("all entries have sha512");
// Round-trip the Checksums-Sha512 field through the parser.
let mut reparsed = FileChecksums::new();
for (key, entry) in FileChecksums::parse_field(ChecksumKind::Sha512, &sha512_field).unwrap()
{
reparsed.insert_entry(&key, entry);
}
let keys: Vec<&str> = reparsed.iter().map(|(k, _)| k.as_str()).collect();
assert_eq!(keys, vec!["b.txt", "a.txt"], "insertion order preserved");
for (key, e) in cs.iter() {
let got = reparsed.get(key).unwrap();
assert_eq!(got.size, e.size, "{key}");
assert_eq!(got.sha512, e.sha512, "{key}");
}
// Rendering the re-parsed registry yields the same field value.
assert_eq!(reparsed.field_sha512().unwrap(), sha512_field);
// The parser dispatches on `kind`: a Checksums-Sha256 body fills the
// sha256 column, leaving the others (including sha512) unknown.
let (key, entry) =
&FileChecksums::parse_field(ChecksumKind::Sha256, &cs.field_sha256()).unwrap()[0];
assert_eq!(entry.sha256, cs.get(key).unwrap().sha256);
assert!(entry.sha512.is_empty());
let (_, entry) =
&FileChecksums::parse_field(ChecksumKind::Sha512, &sha512_field).unwrap()[0];
assert!(entry.md5.is_empty() && entry.sha1.is_empty() && entry.sha256.is_empty());
assert!(!entry.sha512.is_empty());
}
/// Malformed `Checksums-Sha512` bodies (wrong column count, non-numeric
/// size, wrong digest shape) must be rejected with an error naming the
/// field and the offending line; blank lines are tolerated.
#[test]
fn parse_field_rejects_malformed_lines() {
// 128 lowercase hex characters, as rendered by field_sha512.
let digest = "ab".repeat(64);
// Blank lines are skipped.
let entries =
FileChecksums::parse_field(ChecksumKind::Sha512, &format!("\n {digest} 12 a.txt\n\n"))
.unwrap();
assert_eq!(entries.len(), 1);
assert_eq!(entries[0].0, "a.txt");
assert_eq!(entries[0].1.size, 12);
// 2 columns: missing the name.
let err =
FileChecksums::parse_field(ChecksumKind::Sha512, &format!("{digest} 12")).unwrap_err();
assert!(err.contains("Checksums-Sha512"), "{err}");
assert!(err.contains(&format!("{digest} 12")), "{err}");
// 4 columns.
let err =
FileChecksums::parse_field(ChecksumKind::Sha512, &format!(" {digest} 12 bogus a.txt"))
.unwrap_err();
assert!(err.contains("Checksums-Sha512"), "{err}");
assert!(err.contains("a.txt"), "{err}");
// Non-numeric size.
let err =
FileChecksums::parse_field(ChecksumKind::Sha512, &format!(" {digest} twelve a.txt"))
.unwrap_err();
assert!(err.contains("not a number"), "{err}");
assert!(err.contains("twelve"), "{err}");
// Digest that is not 128 lowercase hex characters.
let err = FileChecksums::parse_field(ChecksumKind::Sha512, " abc123 12 a.txt").unwrap_err();
assert!(err.contains("lowercase hex"), "{err}");
}
/// `field_sha512` is only-if-populated: an entry merged without a SHA-512
/// digest (e.g. taken from a `.dsc`) suppresses the whole field instead
/// of rendering an incomplete checksum list.
#[test]
fn field_sha512_omitted_when_any_digest_missing() {
let dir = tempfile::tempdir().unwrap();
let a = dir.path().join("a.txt");
std::fs::write(&a, b"aaa").unwrap();
let mut cs = FileChecksums::new();
cs.add_file(&a).unwrap();
assert!(cs.field_sha512().is_some());
cs.insert_entry(
"from.dsc",
Entry {
size: 12,
md5: "d41d8cd98f00b204e9800998ecf8427e".to_string(),
sha1: "da39a3ee5e6b4b0d3255bfef95601890afd80709".to_string(),
sha256: format!("e3b0{:0>62}", "0"),
sha512: String::new(), // not recorded in .dsc files
},
);
assert!(
cs.field_sha512().is_none(),
"one incomplete entry must suppress Checksums-Sha512"
);
// The other kinds are unaffected.
assert!(!cs.field_md5().is_empty());
assert!(!cs.field_sha1().is_empty());
assert!(!cs.field_sha256().is_empty());
}
#[test]
fn insertion_order_preserved() {
let dir = tempfile::tempdir().unwrap();
+52 -5
View File
@@ -10,6 +10,9 @@ use std::path::Path;
/// A single deb822 paragraph: an ordered list of `(field, value)` pairs.
///
/// The parser is lenient: duplicate field names are kept as separate entries
/// (accessors see the first one; `set` collapses them back to a single one).
///
/// Values are stored with continuation-line breaks as `\n` and without the
/// leading whitespace of continuation lines. Serialization re-adds a single
/// leading space in front of every continuation line, matching dpkg output;
@@ -27,6 +30,10 @@ impl Paragraph {
}
/// Look up a field value (case-insensitive field name).
///
/// Returns the first match. The parser is lenient and keeps duplicate
/// field names as-is; use [`Paragraph::iter`] to reach the other
/// occurrences. [`Paragraph::set`] collapses them.
pub fn get(&self, field: &str) -> Option<&str> {
self.fields
.iter()
@@ -34,17 +41,26 @@ impl Paragraph {
.map(|(_, v)| v.as_str())
}
/// Set a field value, replacing any previous occurrence (case-insensitive).
/// Appends the field at the end if it did not exist yet.
/// Set a field value, replacing all case-insensitive duplicates: after
/// the call at most one entry with this field name remains — the updated
/// one, kept at its original position. Appends the field at the end if
/// no entry existed yet.
pub fn set(&mut self, field: &str, value: &str) {
for (k, v) in self.fields.iter_mut() {
let mut updated = false;
self.fields.retain_mut(|(k, v)| {
if k.eq_ignore_ascii_case(field) {
if updated {
return false;
}
*v = value.to_string();
return;
}
updated = true;
}
true
});
if !updated {
self.fields.push((field.to_string(), value.to_string()));
}
}
/// Remove a field (case-insensitive). Returns true if it was present.
pub fn remove(&mut self, field: &str) -> bool {
@@ -337,6 +353,37 @@ iQEcBAABCgAGBQJabcdAAoJEL abc
assert_eq!(p.iter().count(), 1);
}
#[test]
fn lenient_parse_keeps_duplicate_fields() {
let paras = parse_paragraphs("Package: hello\nDepends: a\ndepends: b\n");
let p = &paras[0];
// deb822 forbids duplicate fields but the parser is lenient and keeps
// both entries; `get` returns the first.
let depends: Vec<_> = p
.iter()
.filter(|(k, _)| k.eq_ignore_ascii_case("Depends"))
.collect();
assert_eq!(depends, [("Depends", "a"), ("depends", "b")]);
assert_eq!(p.get("Depends"), Some("a"));
}
#[test]
fn set_removes_case_insensitive_duplicates() {
let mut paras = parse_paragraphs("Package: hello\nDepends: a\ndepends: b\n");
let mut p = paras.remove(0);
p.set("Depends", "c");
// Exactly one depends-family entry remains, with the new value.
let depends: Vec<_> = p
.iter()
.filter(|(k, _)| k.eq_ignore_ascii_case("Depends"))
.collect();
assert_eq!(depends, [("Depends", "c")]);
assert_eq!(p.get("depends"), Some("c"));
// ...kept at its original position, and a write round-trip no longer
// leaks the stale duplicate.
assert_eq!(write_paragraph(&p), "Package: hello\nDepends: c\n");
}
#[test]
fn remove_field() {
let mut p = Paragraph::new();
+221 -21
View File
@@ -10,7 +10,7 @@
//! (<https://manpages.debian.org/libdpkg-perl>) and were validated
//! differentially against the real tool.
use std::collections::HashMap;
use std::collections::{HashMap, HashSet};
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
@@ -583,6 +583,24 @@ pub struct Facts {
build_arch: String,
installed: HashMap<String, Vec<InstalledPkg>>,
provided: HashMap<String, Vec<ProvidedPkg>>,
/// Virtual packages whose `Provides` entries had to be rejected as a
/// whole (unparseable field, malformed version, or a relation other
/// than `=`), mirroring a dpkg rejection of the entry: versioned
/// relations on them stay undecidable instead of turning unmet.
unreadable_provides: HashSet<String>,
}
/// Best-effort virtual package names mentioned in a raw `Provides` value,
/// used to remember rejected entries whose content cannot even be parsed
/// (the full dependency grammar is deliberately not re-applied here).
fn raw_provides_names(field: &str) -> impl Iterator<Item = &str> {
field
.split([',', '|'])
.filter_map(|alt| alt.split_whitespace().next())
.map(|tok| match tok.split_once(':') {
Some((name, _qual)) => name,
None => tok,
})
}
impl Facts {
@@ -593,6 +611,7 @@ impl Facts {
build_arch: build_arch.to_string(),
installed: HashMap::new(),
provided: HashMap::new(),
unreadable_provides: HashSet::new(),
}
}
@@ -609,6 +628,11 @@ impl Facts {
}
/// Record that `provider` provides the virtual package `virtual_name`.
///
/// A `relation` other than [`Relation::Eq`], or a `version` that fails
/// to parse, makes the provide unusable: versioned relations on
/// `virtual_name` then evaluate to `None` (undecidable) instead of a
/// possibly-wrong `Some(false)`.
pub fn add_provided(
&mut self,
virtual_name: &str,
@@ -639,7 +663,10 @@ impl Facts {
/// Only stanzas whose `Status` ends with `ok installed` participate;
/// their `Provides` field registers versioned/unversioned virtual
/// packages (architecture-restricted provides are reduced against
/// `host_arch`).
/// `host_arch`). A `Provides` field that fails to parse, or that
/// carries a relation other than `=`, is rejected as a whole (like
/// dpkg rejects the entry) and its virtual names are remembered as
/// unreadable.
pub fn from_status(content: &str, host_arch: &str, build_arch: &str) -> Facts {
let mut facts = Facts::new(host_arch, build_arch);
for para in crate::debian::control::parse_paragraphs(content) {
@@ -664,20 +691,38 @@ impl Facts {
union: true,
build_dep: false,
};
// Virtual (Provides) fields only accept '=' relations; a
// parse failure skips the whole field, like dpkg does.
let Ok(parsed) = Deps::parse_inner(provides, &opts, true) else {
// Virtual (Provides) fields only accept versionless or
// exactly '='-versioned alternatives; a field that fails
// to parse, or that carries any other relation, is
// rejected as a whole, like dpkg rejects the entry. The
// mentioned virtual names are remembered so versioned
// relations on them stay undecidable instead of turning
// into a possibly-wrong "unmet" verdict.
let mut parsed = None;
let mut rejected: Vec<String> = Vec::new();
match Deps::parse_inner(provides, &opts, true) {
Ok(deps) => {
if deps.clauses().flatten().any(|alt| {
alt.constraint
.as_ref()
.is_some_and(|c| c.relation != Relation::Eq)
}) {
rejected
.extend(deps.clauses().flatten().map(|alt| alt.package.clone()));
} else {
parsed = Some(deps);
}
}
// The field could not even be parsed: recover the raw
// names so the rejection is still remembered.
Err(_) => rejected.extend(raw_provides_names(provides).map(str::to_string)),
}
facts.unreadable_provides.extend(rejected);
let Some(parsed) = parsed else {
continue;
};
for clause in parsed.clauses() {
for alt in clause {
if alt
.constraint
.as_ref()
.is_some_and(|c| c.relation != Relation::Eq)
{
continue;
}
let provided_version = alt
.constraint
.as_ref()
@@ -755,11 +800,21 @@ impl Facts {
}
}
// A rejected Provides entry for this virtual package carries
// information that could not be read: a versioned relation can
// then not be decided (an unversioned one only needs the name,
// which the rejection removed).
if rel.constraint.is_some() && self.unreadable_provides.contains(&rel.package) {
lackinfos = true;
}
if let Some(providers) = self.provided.get(&rel.package) {
for vp in providers {
// Only unversioned provides and strictly-versioned provides
// can satisfy a dependency.
// Only unversioned provides and exactly-versioned provides
// can satisfy a dependency; anything else is an invalid
// provide and leaves the relation undecidable.
if vp.relation.is_some_and(|r| r != Relation::Eq) {
lackinfos = true;
continue;
}
match &rel.constraint {
@@ -767,11 +822,16 @@ impl Facts {
let Some(vp_version) = &vp.version else {
continue;
};
if let Ok(vp_v) = DebianVersion::parse(vp_version)
&& constraint.relation.eval(&vp_v, &constraint.version)
{
match DebianVersion::parse(vp_version) {
Ok(vp_v) if constraint.relation.eval(&vp_v, &constraint.version) => {
return Some(true);
}
// An unreadable provided version, like an
// unreadable installed version, makes the
// relation undecidable instead of unmet.
Err(_) => lackinfos = true,
Ok(_) => {}
}
}
None => return Some(true),
}
@@ -785,8 +845,15 @@ impl Facts {
/// Options for [`check_build_depends`].
#[derive(Debug, Clone)]
pub struct CheckOpts {
/// Host architecture (defaults to the native architecture).
/// Host architecture, i.e. `DEB_HOST_ARCH`: the architecture the
/// packages are built FOR (defaults to the native architecture).
/// Bracketed `foo [arch]` restrictions evaluate against it.
pub host_arch: String,
/// Build architecture, i.e. `DEB_BUILD_ARCH`: the architecture the
/// build runs ON (defaults to the native architecture). `:native`
/// dependency qualifiers and the dpkg status attribution of the
/// build-side facts resolve against it.
pub build_arch: String,
/// Active build profiles.
pub build_profiles: Vec<String>,
/// Ignore `Build-Depends-Arch`/`Build-Conflicts-Arch` (`-A`).
@@ -804,6 +871,7 @@ impl Default for CheckOpts {
fn default() -> Self {
CheckOpts {
host_arch: arch::native().unwrap_or_default(),
build_arch: arch::native().unwrap_or_default(),
build_profiles: Vec::new(),
ignore_arch: false,
ignore_indep: false,
@@ -904,14 +972,14 @@ pub fn check_build_depends(control: &ControlInfo, opts: &CheckOpts) -> Result<Un
let bc_value = bc_parts.join(", ");
let status_path = opts.admindir.join("status");
let facts = Facts::load_status(&status_path, &opts.host_arch, &opts.host_arch)?;
let facts = Facts::load_status(&status_path, &opts.host_arch, &opts.build_arch)?;
let mut report = UnmetReport::default();
if !bd_value.trim().is_empty() {
let parse_opts = ParseOpts {
host_arch: opts.host_arch.clone(),
build_arch: opts.host_arch.clone(),
build_arch: opts.build_arch.clone(),
build_profiles: opts.build_profiles.clone(),
reduce_restrictions: true,
union: false,
@@ -933,7 +1001,7 @@ pub fn check_build_depends(control: &ControlInfo, opts: &CheckOpts) -> Result<Un
if !bc_value.trim().is_empty() {
let parse_opts = ParseOpts {
host_arch: opts.host_arch.clone(),
build_arch: opts.host_arch.clone(),
build_arch: opts.build_arch.clone(),
build_profiles: opts.build_profiles.clone(),
reduce_restrictions: true,
union: true,
@@ -1243,6 +1311,73 @@ Provides: old-virtual (= 0.5)
assert_eq!(facts.evaluate_relation(&o("old-virtual")), Some(true));
}
/// A `Provides` entry that dpkg would reject as a whole (a malformed
/// provided version, or a relation other than `=`) must not degrade to
/// a bogus "unmet" verdict: versioned relations on the affected virtual
/// names stay undecidable.
#[test]
fn corrupt_provides_are_undecidable_not_unmet() {
// `not-a-version!` is rejected by DebianVersion::parse ('!' is not
// a legal version character), unlike dpkg-invalid but here-valid
// letter-only spellings.
let status = "\
Package: bad-version-provider
Status: install ok installed
Version: 1.0
Architecture: amd64
Provides: virt (= not-a-version!)
Package: bad-relation-provider
Status: install ok installed
Version: 1.0
Architecture: amd64
Provides: virt2 (>= 1.0), plain
";
let facts = Facts::from_status(status, "amd64", "amd64");
let o = |s: &str| parse_simple(s, true).unwrap();
// The provided version fails to parse: undecidable, not unmet.
assert_eq!(facts.evaluate_relation(&o("virt (>= 0.5)")), None);
// A non-'=' relation invalidates the whole Provides field, so
// neither of its alternatives may produce a verdict.
assert_eq!(facts.evaluate_relation(&o("virt2 (>= 0.5)")), None);
assert_eq!(facts.evaluate_relation(&o("plain (>= 0.5)")), None);
// Unversioned relations only need the name, which the rejected
// entries no longer provide: genuinely unmet.
assert_eq!(facts.evaluate_relation(&o("virt")), Some(false));
assert_eq!(facts.evaluate_relation(&o("virt2")), Some(false));
assert_eq!(facts.evaluate_relation(&o("plain")), Some(false));
}
/// The same undecidable verdicts through the direct facts API: an
/// unreadable provided version and an invalid (non-`=`) provide each
/// leave a versioned relation undecided, while a readable provider
/// elsewhere still satisfies it.
#[test]
fn unreadable_provided_version_and_relation_are_undecidable() {
let o = |s: &str| parse_simple(s, true).unwrap();
let mut facts = Facts::new("amd64", "amd64");
facts.add_installed("provider", "1.0", "amd64", "no");
// `virt (= not-a-version!)`: the version string fails to parse.
facts.add_provided(
"virt",
Some(Relation::Eq),
Some("not-a-version!"),
"provider",
);
assert_eq!(facts.evaluate_relation(&o("virt (>= 0.5)")), None);
// `virt2 (>= 1.0)`: a non-'=' provide is invalid.
facts.add_provided("virt2", Some(Relation::Ge), Some("1.0"), "provider");
assert_eq!(facts.evaluate_relation(&o("virt2 (>= 0.5)")), None);
// A readable provider decides the relation when it matches;
// otherwise the unreadable entry keeps it undecidable.
facts.add_provided("virt", Some(Relation::Eq), Some("2.0"), "better");
assert_eq!(facts.evaluate_relation(&o("virt (>= 0.5)")), Some(true));
assert_eq!(facts.evaluate_relation(&o("virt (>> 2.0)")), None);
}
#[test]
fn simplify_reports_unmet() {
let facts = Facts::from_status(STATUS, "amd64", "amd64");
@@ -1363,4 +1498,69 @@ Architecture: amd64
let report = check_build_depends(&control_conflict, &opts).unwrap();
assert_eq!(report.message(), "unmet build conflicts: mypackage");
}
#[test]
fn check_build_depends_cross_native_qualifier() {
let dir = tempfile::tempdir().unwrap();
let admindir = dir.path();
// Cross build: armhf packages built ON an amd64 machine, so
// DEB_HOST_ARCH=armhf but DEB_BUILD_ARCH=amd64.
let opts = CheckOpts {
host_arch: "armhf".to_string(),
build_arch: "amd64".to_string(),
admindir: admindir.to_path_buf(),
..Default::default()
};
let control_for = |bd: &str| {
ControlInfo::parse_content(&format!(
"Source: t\nMaintainer: a <a@b.c>\nBuild-Depends: {bd}\n\nPackage: t\nArchitecture: any\nDescription: x\n y\n"
))
.unwrap()
};
// foo is only installed for the build architecture, like a native
// toolchain package pulled in on the build machine.
std::fs::write(
admindir.join("status"),
"\
Package: foo
Status: install ok installed
Version: 1.0
Architecture: amd64
",
)
.unwrap();
// `:native` resolves against the BUILD architecture: satisfied even
// though the host architecture is armhf.
let report = check_build_depends(&control_for("foo:native"), &opts).unwrap();
assert!(report.is_ok());
// Bracketed architecture restrictions keep evaluating against the
// HOST architecture: the amd64 instance cannot satisfy `foo [armhf]`.
// (The applied restriction reduces away, per dpkg's reduce_arch.)
let report = check_build_depends(&control_for("foo [armhf]"), &opts).unwrap();
assert_eq!(report.message(), "unmet build dependencies: foo");
// Now foo is only installed for the host architecture.
std::fs::write(
admindir.join("status"),
"\
Package: foo
Status: install ok installed
Version: 1.0
Architecture: armhf
",
)
.unwrap();
// `:native` no longer matches: no amd64 instance is installed.
let report = check_build_depends(&control_for("foo:native"), &opts).unwrap();
assert_eq!(report.message(), "unmet build dependencies: foo:native");
// The host-arch restriction matches the armhf instance.
let report = check_build_depends(&control_for("foo [armhf]"), &opts).unwrap();
assert!(report.is_ok());
}
}
+1 -1
View File
@@ -23,7 +23,7 @@ pub use changelog::{
ChangelogEntry, parse_changelog_entry, parse_changelog_entry_from_str,
parse_previous_version_from_str,
};
pub use checksums::{Entry as ChecksumEntry, FileChecksums};
pub use checksums::{ChecksumKind, Entry as ChecksumEntry, FileChecksums};
pub use control::{
ControlInfo, Paragraph, parse_paragraphs, strip_clearsigned_armour, write_paragraph,
};
+27
View File
@@ -48,6 +48,12 @@ impl DebianVersion {
}
}
if let Some(rev) = &debian_revision {
if rev.is_empty() {
// dpkg rejects a trailing hyphen: "bad syntax: revision
// number is empty". Native versions (no `-` at all) are
// handled above and stay valid.
return Err(format!("empty debian revision in '{}'", raw));
}
for c in rev.chars() {
if !(c.is_ascii_alphanumeric() || matches!(c, '.' | '+' | '~') || !c.is_ascii()) {
return Err(format!(
@@ -329,6 +335,27 @@ mod tests {
assert!(DebianVersion::parse("1.0~rc1-2").is_ok());
}
/// dpkg rejects a trailing `-` ("bad syntax: revision number is
/// empty") but accepts `1.0--1`, where the revision is the text after
/// the *last* hyphen (upstream `1.0-` + revision `1`).
#[test]
fn version_empty_revision() {
let err = DebianVersion::parse("1.0-").unwrap_err();
assert!(err.contains("empty"), "unexpected message: {err}");
assert!(DebianVersion::parse("1.0-").is_err());
// Epoch variants take the same path.
assert!(DebianVersion::parse("3:1.0-").is_err());
assert!(DebianVersion::parse("1.0-1").is_ok());
// Native versions (no revision at all) are still fine.
assert!(DebianVersion::parse("1.0").is_ok());
assert!(DebianVersion::parse("3:1.0").is_ok());
let v = DebianVersion::parse("1.0--1").unwrap();
assert_eq!(v.upstream, "1.0-");
assert_eq!(v.debian_revision.as_deref(), Some("1"));
}
fn cmp_sign(a: &DebianVersion, b: &DebianVersion) -> i32 {
match a.cmp(b) {
std::cmp::Ordering::Less => -1,
+48
View File
@@ -53,9 +53,14 @@ lazy_static! {
// Shared HTTP client used for all outgoing plain requests: timeouts keep
// a hanging remote (connect or transfer) from stalling pkh indefinitely.
// The short pool idle timeout and TCP keepalive avoid reusing keep-alive
// connections that the remote closed in the meantime, which surfaces as
// spurious 'error sending request' failures on busy mirrors/CDNs.
static ref HTTP_CLIENT: reqwest::Client = reqwest::Client::builder()
.connect_timeout(Duration::from_secs(10))
.timeout(Duration::from_secs(30))
.pool_idle_timeout(Duration::from_secs(10))
.tcp_keepalive(Duration::from_secs(30))
.build()
.expect("building the shared HTTP client with static options cannot fail");
}
@@ -66,6 +71,49 @@ pub(crate) fn http_client() -> &'static reqwest::Client {
&HTTP_CLIENT
}
/// GET `url` with bounded retries on transient transport errors (a pooled
/// keep-alive connection closed by the remote, a momentary network hiccup,
/// ...): these always succeed again on a fresh connection, and mirrors are
/// busy enough that unguarded single attempts make bulk operations flaky.
///
/// The response status is not inspected: 404s and the like are meaningful
/// answers, not transport failures.
pub(crate) async fn http_get_retried(url: &str) -> reqwest::Result<reqwest::Response> {
http_get_retried_with_timeout(url, None).await
}
/// [`http_get_retried`] with a per-request timeout override, for large
/// streaming downloads that exceed the shared client's total timeout
pub(crate) async fn http_get_retried_with_timeout(
url: &str,
timeout: Option<Duration>,
) -> reqwest::Result<reqwest::Response> {
const ATTEMPTS: u32 = 3;
let mut last_error: Option<reqwest::Error> = None;
for attempt in 0..ATTEMPTS {
let mut request = http_client().get(url);
if let Some(timeout) = timeout {
request = request.timeout(timeout);
}
match request.send().await {
Ok(response) => return Ok(response),
Err(e) => {
if attempt + 1 < ATTEMPTS {
log::debug!(
"GET '{url}' failed (attempt {}/{}, retrying): {}",
attempt + 1,
ATTEMPTS,
e
);
tokio::time::sleep(Duration::from_millis(300 * (u64::from(attempt) + 1))).await;
}
last_error = Some(e);
}
}
}
Err(last_error.expect("at least one attempt was made"))
}
/// Parse an optional '%Y-%m-%d' date from a CSV cell, warning instead of
/// panicking on invalid remote data
fn parse_optional_date(value: Option<&str>, series: &str, field: &str) -> Option<NaiveDate> {
+4
View File
@@ -37,6 +37,10 @@ pub mod ui;
/// Handle context for .deb building: locally, over ssh, in a chroot...
pub mod context;
/// Quiet test runs: per-test log files, subprocess capture and failure
/// matrix (inert passthrough outside test binaries)
pub(crate) mod test_support;
/// Utility functions
pub(crate) mod utils;
+11 -9
View File
@@ -306,17 +306,19 @@ fn copyright(opts: &NewOptions) -> OutputFile {
OutputFile::new("debian/copyright", out)
}
/// `debian/.gitignore`: the debhelper build artifacts.
/// `debian/.gitignore`: the debhelper build artifacts. The patterns are
/// relative to `debian/` itself (a `debian/`-prefixed pattern would be
/// anchored to `debian/debian/` inside this file, per gitignore(5)).
fn debian_gitignore(opts: &NewOptions) -> OutputFile {
OutputFile::new(
"debian/.gitignore",
format!(
"debian/files\n\
debian/.debhelper/\n\
debian/*.log\n\
debian/{}/\n\
debian/debhelper-build-stamp\n\
debian/*.substvars\n",
"files\n\
.debhelper/\n\
*.log\n\
{}/\n\
debhelper-build-stamp\n\
*.substvars\n",
opts.name
),
)
@@ -766,8 +768,8 @@ mod tests {
let g = super::debian_gitignore(&opts());
assert_eq!(
g.contents,
"debian/files\ndebian/.debhelper/\ndebian/*.log\ndebian/mytool/\n\
debian/debhelper-build-stamp\ndebian/*.substvars\n"
"files\n.debhelper/\n*.log\nmytool/\n\
debhelper-build-stamp\n*.substvars\n"
);
}
+6 -1
View File
@@ -331,7 +331,12 @@ mod tests {
) -> Result<ScaffoldOutcome, Box<dyn Error>> {
let previous = std::env::current_dir()?;
std::env::set_current_dir(dir)?;
let result = scaffold(opts, &MultiProgress::new());
// Hidden draw target: in tests the spinner would redraw from its
// steady-tick thread straight to the real stderr
let result = scaffold(
opts,
&MultiProgress::with_draw_target(crate::ui::progress_draw_target()),
);
std::env::set_current_dir(previous)?;
result
}
+59
View File
@@ -473,6 +473,29 @@ pub fn sanitize_name(input: &str) -> Option<String> {
Some(out)
}
/// Validate the installed command (binary) name: a non-empty, ASCII-only,
/// lowercase identifier (`^[a-z0-9][a-z0-9+.\-_]*$`), matching what
/// dpkg/devscripts accept for executable names in practice. The name is
/// interpolated verbatim into `debian/install`, `debian/rules`, the
/// autopkgtest smoke test, automake variables (`{command}_SOURCES`),
/// `meson.build` and the `[project.scripts]` table, so uppercase letters,
/// whitespace, quotes and shell metacharacters are rejected here instead of
/// generating broken install lines and build files.
pub fn validate_command(command: &str) -> Result<(), String> {
static COMMAND_REGEX: std::sync::OnceLock<Regex> = std::sync::OnceLock::new();
let regex = COMMAND_REGEX.get_or_init(|| Regex::new(r"^[a-z0-9][a-z0-9+.\-_]*$").unwrap());
if command.is_empty() {
return Err("the command name must not be empty".to_string());
}
if !regex.is_match(command) {
return Err(format!(
"'{command}' is not a valid command name: commands must be a \
lowercase identifier (letters, digits, + - . _), e.g. 'mytool'"
));
}
Ok(())
}
/// Validate an upstream version: it must start with a digit (dpkg
/// recommendation, enforced here) and survive [`DebianVersion::parse`] once
/// composed with the Debian revision. It must not contain `-` (the revision
@@ -763,6 +786,9 @@ pub async fn resolve(cli: NewCli) -> Result<NewOptions, String> {
let license = License::parse(cli.license.as_deref().unwrap_or("unknown"));
let command = cli.command.unwrap_or_else(|| name.clone());
// The default is the already-validated package name (a subset of the
// command charset), so only an explicit --command/wizard answer can fail.
validate_command(&command)?;
let maintainer = match &cli.maintainer {
Some(m) => parse_maintainer(m)?,
@@ -988,6 +1014,39 @@ mod tests {
}
}
/// The command name is interpolated verbatim into debian/install,
/// debian/rules, the smoke test, automake variables, meson.build and
/// [project.scripts], so only the safe identifier charset passes.
#[test]
fn command_validator() {
for valid in [
"mytool",
"my.tool", // automake/TOML-friendly spellings in use in Debian
"my+tool",
"my_tool",
"my-tool",
"2ping", // leading digit
"a", // single character
"a1.b+c-d_e",
] {
assert!(validate_command(valid).is_ok(), "{valid} must pass");
}
for invalid in [
"my tool", // whitespace breaks install lines
"a\"b", // quotes break shell snippets
"MyTool", // uppercase
"-lead", // bad first char
"_lead", // bad first char
".dot", // bad first char
"café", // non-ASCII
"a:b", // shell metacharacter
"a/b", // path separator
"", // empty
] {
assert!(validate_command(invalid).is_err(), "{invalid} must fail");
}
}
#[test]
fn sanitize_name_derives_valid_names() {
assert_eq!(sanitize_name("My Tool"), Some("my-tool".to_string()));
+285 -36
View File
@@ -11,7 +11,8 @@
//! travels in the component tarball instead and can be regenerated
//! independently of the upstream sources.
use std::io::{Read, Write};
use std::ffi::OsString;
use std::io::Read;
use std::path::{Path, PathBuf};
use std::process::{Command, Stdio};
use std::time::Duration;
@@ -224,8 +225,16 @@ fn git_archive_tarball(
.into());
}
// Any failure after the destination file was created (spawn error,
// broken pipe, failed git) leaves behind an empty or half-written xz
// file that is not a valid tarball: it is removed before returning.
let outcome = (|| {
let file = std::fs::File::create(&dest)?;
let mut encoder = XzEncoder::new(file, 6);
// git's stderr is inherited, not piped: nothing here drains a piped
// stderr, and a chatty git filling the 64 KiB pipe buffer would
// deadlock the archive — its diagnostics belong on the terminal
// anyway, like every other child process in this module.
let mut child = Command::new("git")
.args([
"archive",
@@ -235,21 +244,29 @@ fn git_archive_tarball(
])
.current_dir(repo)
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.map_err(|e| format!("failed to run 'git archive': {e}"))?;
let mut stdout = child
.stdout
.take()
.ok_or_else(|| "git archive produced no output".to_string())?;
std::io::copy(&mut stdout, &mut encoder)
.map_err(|e| format!("cannot pipe git archive into '{}': {e}", dest.display()))?;
let copied = std::io::copy(&mut stdout, &mut encoder)
.map_err(|e| format!("cannot pipe git archive into '{}': {e}", dest.display()));
// The child is reaped whatever happened to the pipe, and a copy
// error only surfaces once it has been waited on. Closing our end
// first unblocks a git still writing into the pipe (SIGPIPE);
// otherwise `wait` could hang on it forever.
drop(stdout);
let status = child.wait()?;
copied?;
encoder
.finish()
.map_err(|e| format!("failed to write '{}': {e}", dest.display()))?;
let status = child.wait()?;
if !status.success() {
// The half-written xz file is not a valid tarball: remove it.
Ok(status)
})();
match outcome {
Ok(status) if status.success() => {}
Ok(status) => {
let _ = std::fs::remove_file(&dest);
return Err(format!(
"'git archive --format=tar {}' failed with status: {status} \
@@ -258,6 +275,11 @@ fn git_archive_tarball(
)
.into());
}
Err(error) => {
let _ = std::fs::remove_file(&dest);
return Err(error);
}
}
log::info!(
"Created orig tarball from git archive of {tag}: {}",
@@ -339,28 +361,44 @@ fn fetch_and_repack(
/// blocking client must not run on a tokio worker thread (scaffold is
/// called from inside the async runtime), so the download runs on a plain
/// dedicated thread.
///
/// Only the connection carries a timeout (10 s, like the rest of the
/// codebase): a whole-request timeout would cap the ENTIRE download and
/// always fail large release tarballs on slow links. The body is instead
/// streamed to the temporary file chunk by chunk as it arrives, never
/// buffered whole in memory.
fn download_to_temp(url: &str) -> Result<PathBuf, Box<dyn std::error::Error>> {
let url = url.to_string();
let contents = std::thread::spawn(move || {
reqwest::blocking::Client::builder()
.timeout(Duration::from_secs(30))
let download = move || -> Result<PathBuf, Box<dyn std::error::Error + Send + Sync>> {
let mut response = reqwest::blocking::Client::builder()
.connect_timeout(Duration::from_secs(10))
.build()?
.get(&url)
.send()?
.error_for_status()?
.bytes()
})
.join()
.map_err(|_| -> Box<dyn std::error::Error> { "the download thread panicked".into() })??;
.error_for_status()?;
let temp = std::env::temp_dir().join(format!(
"pkh-orig-{}-{}",
std::process::id(),
chrono::Utc::now().timestamp_nanos_opt().unwrap_or_default()
));
let mut file = std::fs::File::create(&temp)?;
file.write_all(&contents)?;
if let Err(error) = response.copy_to(&mut file) {
// A failed download must not leave a partial temporary behind.
let _ = std::fs::remove_file(&temp);
return Err(error.into());
}
Ok(temp)
};
let downloaded: Result<PathBuf, Box<dyn std::error::Error + Send + Sync>> =
std::thread::spawn(download)
.join()
.map_err(|_| -> Box<dyn std::error::Error> { "the download thread panicked".into() })?;
// The thread must carry a `Send + Sync` error box; the auto traits are
// dropped on the way out.
match downloaded {
Ok(temp) => Ok(temp),
Err(error) => Err(error),
}
}
/// Container/compression format of a tarball, detected from its content
@@ -507,12 +545,18 @@ fn repack_tarball_file(
};
let result = repack_tar_stream(reader, name, upstream_version, &dest);
if let Some(mut child) = bzip2_child
&& let Ok(dest_path) = &result
{
if let Some(mut child) = bzip2_child {
if result.is_err() {
// The repack failed and nothing reads the child's stdout
// anymore: kill it (harmless if it already exited — bzip2 may
// be blocked writing into the unread pipe) and reap it,
// instead of leaking a zombie holding an open pipe.
let _ = child.kill();
let _ = child.wait();
} else {
let status = child.wait()?;
if !status.success() {
let _ = std::fs::remove_file(dest_path);
let _ = std::fs::remove_file(&dest);
return Err(format!(
"'bzip2 -dc {}' failed with status: {status}",
path.display()
@@ -520,6 +564,7 @@ fn repack_tarball_file(
.into());
}
}
}
if result.is_err() {
// A failed repack must not leave a half-written tarball behind.
let _ = std::fs::remove_file(&dest);
@@ -527,10 +572,34 @@ fn repack_tarball_file(
result
}
/// How the source tarball organizes its entries, resolved lazily from its
/// leading entries.
#[derive(Default)]
enum SourceLayout {
/// No entry has revealed the layout yet.
#[default]
Undecided,
/// A lone top-level directory entry, held back (name and header) until
/// the next entry either confirms it as the source tarball's own
/// top-level directory (classic layout) or proves the archive flat.
Probation(OsString, Box<tar::Header>),
/// Classic layout: every entry nests under one top-level directory,
/// whose component is stripped from the repacked paths.
Nested(OsString),
/// Flat layout (`tar czf up.tar.gz file1 src/ ...`): entries already
/// sit at the top level and keep their whole path under the prefix.
Flat,
}
/// Rewrite every entry of the tar `stream` under the `<name>-<uver>/`
/// top-level directory (whatever prefix the source tarball used) into the
/// xz-compressed tarball at `dest`. `.git` directories and tar metadata
/// leftovers are dropped, modes travel through.
/// top-level directory into the xz-compressed tarball at `dest`, whatever
/// the source tarball's layout. `.git` directories and tar metadata
/// leftovers are dropped, modes travel through. Classic archives nest
/// everything under one `pkg-1.0/` directory, which is stripped; FLAT
/// archives (`tar czf up.tar.gz file1 src/`) have no such directory, and
/// the old strip-first rule dropped their entries one and all, silently
/// writing an accepted-but-empty orig: flat entries now keep their whole
/// path under the new prefix.
fn repack_tar_stream(
stream: Box<dyn Read>,
name: &str,
@@ -543,26 +612,99 @@ fn repack_tar_stream(
let prefix = format!("{name}-{upstream_version}");
let mut archive = tar::Archive::new(stream);
for entry in archive.entries()? {
let mut layout = SourceLayout::Undecided;
'entries: for entry in archive.entries()? {
let mut entry = entry?;
let original = entry.path()?.to_path_buf();
// Drop the source tarball's top-level directory...
let rest: PathBuf = original
.components()
.skip(1)
.filter(|component| component.as_os_str() != ".git")
.collect();
// ...skipping the top-level entry itself and any entry living
// inside a dropped directory (empty `rest` after a `.git` strip).
if rest.as_os_str().is_empty() {
continue;
}
// GNU tar's pax metadata leftover never travels.
if original
.file_name()
.is_some_and(|name| name == "pax_global_header")
{
continue;
}
// `.git` components are dropped wherever they appear.
let components: Vec<&std::ffi::OsStr> = original
.components()
.filter(|component| component.as_os_str() != ".git")
.map(|component| component.as_os_str())
.collect();
let Some(&first) = components.first() else {
// Nothing left, e.g. the `.git` directory entry itself:
// dropped like everything that lived inside it.
continue;
};
let tail_is_empty = components.len() == 1;
let is_dir = entry.header().entry_type() == tar::EntryType::Directory;
// The path the entry keeps under the new top-level directory: the
// source's own top-level component is stripped in the classic
// nested layout, everything else keeps its whole path.
let rest: PathBuf = 'resolve: {
match std::mem::take(&mut layout) {
SourceLayout::Undecided if tail_is_empty && is_dir => {
// A lone top-level directory entry: hold it back until
// the next entry shows whether it is the source
// tarball's own top-level directory (to strip, the
// classic `pkg-1.0/` layout) or a flat archive's
// top-level directory (to keep).
layout = SourceLayout::Probation(
first.to_os_string(),
Box::new(entry.header().clone()),
);
continue 'entries;
}
SourceLayout::Undecided if tail_is_empty => {
// A top-level file: the archive is flat.
layout = SourceLayout::Flat;
break 'resolve components.iter().collect();
}
SourceLayout::Undecided => {
// Entries nested right away, with no top-level
// directory entry: the first component names the
// source's own top-level directory.
layout = SourceLayout::Nested(first.to_os_string());
break 'resolve components[1..].iter().collect();
}
SourceLayout::Probation(claimed, mut header) => {
if first == claimed.as_os_str() && (is_dir || !tail_is_empty) {
// Confirmed: the held entry is the source tarball's
// own top-level directory, skipped as always.
layout = SourceLayout::Nested(claimed);
if tail_is_empty {
continue 'entries;
}
break 'resolve components[1..].iter().collect();
}
// Refuted: a loose file or a second top-level directory
// proves the archive flat, and the held directory is a
// real one — it travels under the prefix.
let pending = format!("{prefix}/{}", claimed.to_string_lossy());
builder.append_data(&mut header, &pending, std::io::empty())?;
layout = SourceLayout::Flat;
break 'resolve components.iter().collect();
}
SourceLayout::Nested(top) => {
let under_top = first == top.as_os_str();
layout = SourceLayout::Nested(top);
if under_top && tail_is_empty {
// The source top-level directory's own entry.
continue 'entries;
}
break 'resolve if under_top {
components[1..].iter().collect()
} else {
// Sibling top-level content in a nested archive,
// kept whole (the strip-first rule dropped it).
components.iter().collect()
};
}
SourceLayout::Flat => {
layout = SourceLayout::Flat;
break 'resolve components.iter().collect();
}
}
};
let new_path = format!("{prefix}/{}", rest.to_string_lossy());
let mut header = entry.header().clone();
@@ -588,6 +730,14 @@ fn repack_tar_stream(
}
}
// A top-level directory held back that no other entry ever confirmed or
// refuted (the archive holds nothing else): keep it rather than
// silently dropping it.
if let SourceLayout::Probation(claimed, mut header) = layout {
let pending = format!("{prefix}/{}", claimed.to_string_lossy());
builder.append_data(&mut header, &pending, std::io::empty())?;
}
builder
.finish()
.map_err(|e| format!("failed to write '{}': {e}", dest.display()))?;
@@ -597,6 +747,7 @@ fn repack_tar_stream(
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
/// List the entry names of an xz tarball.
fn tarball_names(path: &Path) -> Vec<String> {
@@ -726,6 +877,104 @@ mod tests {
);
}
/// Append a single top-level regular-file entry to a tar builder.
fn append_flat_file<W: std::io::Write>(
builder: &mut tar::Builder<W>,
name: &str,
contents: &str,
) {
let mut header = tar::Header::new_gnu();
header.set_size(contents.len() as u64);
header.set_mode(0o644);
header.set_cksum();
builder
.append_data(&mut header, name, contents.as_bytes())
.unwrap();
}
/// Append a single top-level directory entry to a tar builder.
fn append_flat_dir<W: std::io::Write>(builder: &mut tar::Builder<W>, name: &str) {
let mut header = tar::Header::new_gnu();
header.set_entry_type(tar::EntryType::Directory);
header.set_size(0);
header.set_mode(0o755);
header.set_cksum();
builder
.append_data(&mut header, name, std::io::empty())
.unwrap();
}
/// Regression (flat tarball origin): `tar czf up.tar.gz file1 file2`
/// archives its entries at the TOP level, with no leading directory.
/// The old strip-first-component rule reduced every entry to nothing
/// and silently wrote an accepted-but-empty orig. Flat entries must
/// travel under the canonical `<name>-<uver>/` top-level directory,
/// and no entry may remain outside it.
#[test]
fn repack_puts_flat_tarball_entries_under_the_prefix() {
let dir = tempfile::tempdir().unwrap();
let tree = dir.path().join("mytool");
std::fs::create_dir_all(&tree).unwrap();
let source = dir.path().join("flat.tar.gz");
{
let file = std::fs::File::create(&source).unwrap();
let encoder = flate2::write::GzEncoder::new(file, flate2::Compression::fast());
let mut builder = tar::Builder::new(encoder);
append_flat_file(&mut builder, "file1", "one\n");
append_flat_file(&mut builder, "file2", "two\n");
builder.into_inner().unwrap();
}
let dest = repack_tarball_file(&source, "mytool", "1.0.0", &tree).unwrap();
let names = tarball_names(&dest);
assert!(names.iter().any(|n| n == "mytool-1.0.0/file1"), "{names:?}");
assert!(names.iter().any(|n| n == "mytool-1.0.0/file2"), "{names:?}");
// No entry may stay loose at the top level.
assert!(
names.iter().all(|n| n.starts_with("mytool-1.0.0/")),
"{names:?}"
);
}
/// A flat archive may also carry top-level DIRECTORIES: with `src/`,
/// `file1` and `src/main.rs` all at the top level, every path keeps its
/// shape under the prefix (`src/` → `<topdir>/src/`). The directory
/// entry comes FIRST here, so the held-back-entry mechanism must
/// refute it as the source's own top-level directory for `src/` to
/// survive — a classic nested `pkg-1.0/` first entry is still skipped,
/// as the other repack tests assert.
#[test]
fn repack_keeps_flat_top_level_directories_under_the_prefix() {
let dir = tempfile::tempdir().unwrap();
let tree = dir.path().join("mytool");
std::fs::create_dir_all(&tree).unwrap();
let source = dir.path().join("flat-with-dir.tar.gz");
{
let file = std::fs::File::create(&source).unwrap();
let encoder = flate2::write::GzEncoder::new(file, flate2::Compression::fast());
let mut builder = tar::Builder::new(encoder);
append_flat_dir(&mut builder, "src");
append_flat_file(&mut builder, "file1", "one\n");
append_flat_file(&mut builder, "src/main.rs", "code\n");
builder.into_inner().unwrap();
}
let dest = repack_tarball_file(&source, "mytool", "2.0.0", &tree).unwrap();
let names = tarball_names(&dest);
assert!(names.iter().any(|n| n == "mytool-2.0.0/src"), "{names:?}");
assert!(
names.iter().any(|n| n == "mytool-2.0.0/src/main.rs"),
"{names:?}"
);
assert!(names.iter().any(|n| n == "mytool-2.0.0/file1"), "{names:?}");
assert!(
names.iter().all(|n| n.starts_with("mytool-2.0.0/")),
"{names:?}"
);
}
#[test]
fn repack_drops_git_dirs_and_unsupported_entries() {
let dir = tempfile::tempdir().unwrap();
+429 -91
View File
@@ -102,12 +102,8 @@ fn is_interactive() -> bool {
/// answer is skipped (flag > detected/probe > default merge order).
async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
let cwd = std::env::current_dir()?;
let detect_dir = cli.source.clone().unwrap_or_else(|| cwd.clone());
let detection = detect::detect(&detect_dir);
let probe = match &detection {
Detection::Single(id) => templates::get(*id).and_then(|t| t.probe(&detect_dir)),
_ => None,
};
let mut detect_dir = cli.source.clone().unwrap_or_else(|| cwd.clone());
let (detection, mut probe) = detect_and_probe(&detect_dir);
// Whether the flags imply a fresh skeleton (name given, no --source).
let implied_skeleton = cli.source.is_none() && cli.name.is_some();
// Whether the language question is skipped by a confident detection
@@ -122,10 +118,30 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
cli.name = Some(answer);
}
// 2. Language / build system.
let mut preselected: Option<TemplateId> = None;
match &detection {
Detection::Single(id) if detection_decides => {
// 2. Language / build system. An explicit `--lang` wins over any
// detection (flag > detected/probe > default): it is never
// overwritten and the question is never re-asked — the detection is
// only logged for information.
match language_choice(cli.lang.as_deref(), &detection, detection_decides) {
LanguageChoice::Flag => match &detection {
Detection::Single(id) => log::info!(
"Detected: {} project in {}; --lang takes precedence",
id.display_name(),
detect_dir.display()
),
Detection::Ambiguous(candidates) => log::info!(
"Several build systems found in {} ({}); --lang takes \
precedence",
detect_dir.display(),
candidates
.iter()
.map(|id| id.as_str())
.collect::<Vec<_>>()
.join(", ")
),
Detection::Empty => {}
},
LanguageChoice::Detected(id) => {
log::info!(
"Detected: {} project in {}",
id.display_name(),
@@ -133,12 +149,16 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
);
cli.lang = Some(id.as_str().to_string());
}
Detection::Single(id) => {
// A skeleton was asked for: still ask, preselecting the
// detected ecosystem.
preselected = Some(*id);
LanguageChoice::Ask(preselected) => {
let menu = language_menu(&[]);
let default = preselected
.unwrap_or(TemplateId::Empty)
.display_name()
.to_string();
let id = select_template(&menu, &default)?;
cli.lang = Some(id.as_str().to_string());
}
Detection::Ambiguous(candidates) => {
LanguageChoice::Ambiguous(candidates) => {
log::info!(
"Several build systems found in {} ({}): candidates listed \
first, the highest-precedence one preselected",
@@ -149,20 +169,10 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
.collect::<Vec<_>>()
.join(", ")
);
let menu = language_menu(candidates);
let menu = language_menu(&candidates);
let id = select_template(&menu, &menu[0])?;
cli.lang = Some(id.as_str().to_string());
}
Detection::Empty => {}
}
if cli.lang.is_none() {
let menu = language_menu(&[]);
let default = preselected
.unwrap_or(TemplateId::Empty)
.display_name()
.to_string();
let id = select_template(&menu, &default)?;
cli.lang = Some(id.as_str().to_string());
}
let template = TemplateId::parse(cli.lang.as_deref().unwrap_or_default())?;
@@ -235,8 +245,31 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
let packaged_dir = cli.source.clone();
let mut origin = packaged_dir.as_deref().and_then(GitOrigin::detect);
// The probe data read above describes `detect_dir` — the cwd unless
// `--source` pointed somewhere. When the source-location answer just
// redirected the wizard to another directory, re-run the detection +
// probe there (the same code path as the initial pass) so every
// following probe-derived default — upstream version, description,
// homepage, license, command, the pkg-config hint — describes the
// project actually being packaged. The detected directory itself is
// never re-probed (the data is already fresh and the scan is not free),
// and explicit flags keep winning throughout: the probe only ever feeds
// the defaults of questions without a flag answer.
if let Some(chosen) = packaged_dir.as_deref()
&& !same_directory(chosen, &detect_dir)
{
let (_, refreshed) = detect_and_probe(chosen);
probe = refreshed;
// The license sniff and the pkg-config hint read the same directory
// the probe data now comes from.
detect_dir = chosen.to_path_buf();
}
// 4. Upstream version: probed project version, then the tag HEAD sits
// on, then `<lasttag>+git<YYYYMMDD>.<hash>`, then 0.1.0.
// on, then `<lasttag>+git<YYYYMMDD>.<hash>`, then 0.1.0. A raw probe
// that fails `validate_upstream_version` (e.g. `1.0-2` or `v1.0`) is
// not offered as the default — the same validator `resolve` applies
// decides, so Enter can never accept it and crash late.
if cli.upstream_version.is_none() {
let default = probe
.as_ref()
@@ -245,9 +278,8 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
.or_else(|| origin.as_ref().and_then(GitOrigin::git_version))
.unwrap_or_else(|| "0.1.0".to_string());
let revision = cli.revision.unwrap_or(1);
let answer = ask_text("Upstream version", &default, move |version: &str| {
options::validate_upstream_version(version, revision)
})?;
let validate = move |version: &str| options::validate_upstream_version(version, revision);
let answer = ask_text("Upstream version", &default, validate)?;
cli.upstream_version = Some(answer.clone());
// The typed version names an existing tag HEAD is not on: offer to
@@ -320,31 +352,25 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
.as_ref()
.and_then(|p| p.description.clone())
.unwrap_or_default();
loop {
let answer = ask_text(
"One-line description",
&default,
required_answer("the description"),
)?;
if !answer.trim().is_empty() {
let validate = required_answer("the description");
let answer = ask_text("One-line description", &default, validate)?;
cli.description = Some(answer);
break;
}
log::warn!("A one-line description is required to scaffold a package");
}
}
// 7. Homepage.
// 7. Homepage (blank skips; a probed default must still be a valid URL).
if cli.homepage.is_none() {
let default = probe
.as_ref()
.and_then(|p| p.homepage.clone())
.unwrap_or_default();
let answer = ask_text(
"Homepage (blank to skip)",
&default,
options::validate_homepage,
)?;
let validate = |url: &str| {
if url.is_empty() {
Ok(())
} else {
options::validate_homepage(url)
}
};
let answer = ask_text("Homepage (blank to skip)", &default, validate)?;
if !answer.is_empty() {
cli.homepage = Some(answer);
}
@@ -364,17 +390,8 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
options.contains(&answer.to_string())
})?;
if answer == LICENSE_OTHER {
let license = loop {
let candidate = ask_text(
"License (SPDX identifier)",
&custom_default,
required_answer("the license identifier"),
)?;
if !candidate.trim().is_empty() {
break candidate;
}
log::warn!("A license identifier is required when picking the free-text entry");
};
let validate = required_answer("the license identifier");
let license = ask_text("License (SPDX identifier)", &custom_default, validate)?;
cli.license = Some(license);
} else {
cli.license = Some(answer);
@@ -382,43 +399,42 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
}
// 9. Command name (skipped for the empty template, where nothing is
// installed).
// installed). Typed answers and the offered default go through the
// same `validate_command` bar as `resolve` applies (which also
// requires a non-empty answer), so an unusable probe is withheld and
// invalid input re-asks here instead of failing late in `resolve`.
if cli.command.is_none() && template != TemplateId::Empty {
let default = probe
.as_ref()
.and_then(|p| p.command.clone())
.unwrap_or_else(|| cli.name.clone().unwrap_or_default());
let command = ask_text(
"Command name",
&default,
required_answer("the command name"),
)?;
cli.command = Some(if command.is_empty() {
cli.name.clone().unwrap_or_default()
} else {
command
});
let command = ask_text("Command name", &default, options::validate_command)?;
cli.command = Some(command);
}
// 10. Maintainer, defaulting to the DEBEMAIL/git-config identity (an
// empty answer re-asks).
// 10. Maintainer, defaulting to the DEBEMAIL/git-config identity. The
// git-derived default goes through `parse_maintainer` like typed
// input: an empty git user.email yields a `Name <>` default that is
// ignored (the question is asked without one) instead of accepted
// verbatim and only failing late in `resolve`. An empty answer
// re-asks.
if cli.maintainer.is_none() {
let default = crate::changelog::get_maintainer_info()
let candidate = crate::changelog::get_maintainer_info()
.map(|(name, email)| format!("{name} <{email}>"))
.unwrap_or_default();
let maintainer = loop {
let answer = ask_text("Maintainer", &default, |answer: &str| {
options::parse_maintainer(answer).map(|_| ())
})?;
if !answer.is_empty() {
break answer;
}
let default = if options::parse_maintainer(&candidate).is_ok() {
candidate
} else {
String::new()
};
if default.is_empty() {
log::warn!(
"Could not determine a maintainer default (no DEBFULLNAME/\
DEBEMAIL and no git user config): answer as 'Name <email>'"
);
};
cli.maintainer = Some(maintainer);
}
let validate = |answer: &str| options::parse_maintainer(answer).map(|_| ());
cli.maintainer = Some(ask_text("Maintainer", &default, validate)?);
}
// 11. Target distribution.
@@ -458,10 +474,11 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
// 13. Metapackage Depends (empty template only).
if template == TemplateId::Empty && cli.depends.is_empty() {
let validate = |answer: &str| options::validate_depends(answer).map(|_| ());
let answer = ask_text(
"Depends (metapackage, comma-separated, blank for an empty base)",
"",
|answer: &str| options::validate_depends(answer).map(|_| ()),
validate,
)?;
if !answer.trim().is_empty() {
cli.depends = vec![answer];
@@ -636,6 +653,33 @@ pub async fn offer_verification(
}
}
/// Detection + template probe of one candidate directory: the probe data
/// exists only for a confidently detected single-template project. The one
/// code path for both wizard passes — the initial sweep of the detected
/// directory and the re-probe when the source-location answer redirects the
/// wizard elsewhere.
fn detect_and_probe(dir: &std::path::Path) -> (Detection, Option<ProbeResult>) {
let detection = detect::detect(dir);
let probe = match &detection {
Detection::Single(id) => templates::get(*id).and_then(|t| t.probe(dir)),
_ => None,
};
(detection, probe)
}
/// Whether two paths name the same directory: lexical equality first, then
/// (for spellings like `.` next to the absolute cwd) the canonicalized
/// forms. A path that cannot be canonicalized only ever equals itself.
fn same_directory(a: &std::path::Path, b: &std::path::Path) -> bool {
if a == b {
return true;
}
match (a.canonicalize(), b.canonicalize()) {
(Ok(a), Ok(b)) => a == b,
_ => false,
}
}
/// The preselected default of the pkg-config opt-in question: whether the
/// project's build file hints at pkg-config usage (`dependency(` in
/// meson.build, `pkg_check_modules` / `find_package(PkgConfig` in
@@ -690,6 +734,42 @@ fn language_menu(candidates: &[TemplateId]) -> Vec<String> {
.collect()
}
/// What the language step does with the detection result: how an explicit
/// `--lang` flag combines with it (flag > detected/probe > default).
#[derive(Debug, PartialEq, Eq)]
enum LanguageChoice {
/// An explicit `--lang` wins: it is kept as-is, the question is never
/// re-asked and the detection stays informational.
Flag,
/// No flag, confident detection, packaging the detected directory: the
/// detection decides.
Detected(TemplateId),
/// Ask the question, preselecting the detected ecosystem (a skeleton
/// was asked for); `None` when nothing was detected.
Ask(Option<TemplateId>),
/// Ask the question with the ambiguous candidates listed first (the
/// highest-precedence one preselected).
Ambiguous(Vec<TemplateId>),
}
/// The language step's decision for one wizard run. With the flag absent
/// this mirrors the historical behavior exactly; the flag always wins.
fn language_choice(
flag: Option<&str>,
detection: &Detection,
detection_decides: bool,
) -> LanguageChoice {
if flag.is_some() {
return LanguageChoice::Flag;
}
match detection {
Detection::Single(id) if detection_decides => LanguageChoice::Detected(*id),
Detection::Single(id) => LanguageChoice::Ask(Some(*id)),
Detection::Ambiguous(candidates) => LanguageChoice::Ambiguous(candidates.clone()),
Detection::Empty => LanguageChoice::Ask(None),
}
}
/// The license menu: the curated SPDX list plus the free-text entry.
fn license_menu() -> Vec<String> {
KNOWN_LICENSES
@@ -749,17 +829,54 @@ fn select_from(
}
}
/// The default a question is offered with: a probed default must clear the
/// same `validate` bar as typed input, so an unusable probe (e.g. an
/// upstream version like `1.0-2` carrying a Debian revision) is withheld —
/// the question is asked without a default instead of offering one that
/// Enter would accept verbatim.
fn offered_default<'a>(default: &'a str, validate: &prompt::Validator) -> &'a str {
if default.is_empty() || validate(default).is_ok() {
default
} else {
""
}
}
/// One round of [`ask_text`]: an empty answer takes the (pre-validated)
/// `default`, and whatever answer is finally proposed — typed or the
/// default — must pass `validate`. `Err` carries the validation error so the
/// caller re-asks with it.
fn accept_answer(
answer: &str,
default: &str,
validate: &prompt::Validator,
) -> Result<String, String> {
let answer = if answer.is_empty() { default } else { answer };
validate(answer).map(|_| answer.to_string())
}
/// One free-text question implementing the spec's "Enter accepts the
/// default": an empty answer falls back to `default` (`Esc` keeps its
/// prompt-level meaning of restoring the default). `validate` only ever
/// sees non-empty answers — the empty one is accepted by the prompt loop so
/// it can take the default path; callers that require an answer re-check
/// the result.
/// default": an empty answer falls back to the default (`Esc` keeps its
/// prompt-level meaning of restoring it). Both the typed answer and the
/// default go through `validate`: a probed default that fails validation is
/// never offered ([`offered_default`]), and an empty answer without a valid
/// default is treated like any other invalid answer (re-ask with the
/// validation error) — probe data can never bypass validation and only blow
/// up later in [`options::resolve`].
fn ask_text(
label: &str,
default: &str,
validate: impl Fn(&str) -> Result<(), String> + 'static,
) -> Result<String, Box<dyn Error>> {
// The offered default must clear the same bar as typed input: an
// unusable probe is withheld and the question is asked without one.
let default = offered_default(default, &validate);
// Enter/Esc (and the non-TTY fallback) return the default without
// validation, so pre-decide what an empty answer yields: the default
// itself (it passed the bar above), or the rejection it shares with any
// invalid answer when no valid default exists. Computed here because
// `validate` moves into the prompt wrapper below.
let empty_answer = accept_answer("", default, &validate);
let accept_empty = move |answer: &str| {
if answer.is_empty() {
Ok(())
@@ -767,12 +884,20 @@ fn ask_text(
validate(answer)
}
};
loop {
let answer = prompt::text(label, default, Some(&accept_empty))?;
Ok(if answer.is_empty() {
default.to_string()
// Typed non-empty answers were already validated by the prompt; only
// an empty one resolves to the default, pre-decided above.
let answer = if answer.is_empty() {
empty_answer.clone()
} else {
answer
})
Ok(answer)
};
match answer {
Ok(answer) => return Ok(answer),
Err(error) => log::warn!("{error}"),
}
}
}
/// A validator requiring a non-empty answer.
@@ -1043,6 +1168,59 @@ mod tests {
assert_eq!(unique.len(), menu.len());
}
/// An explicit `--lang` wins over any detection (the flag outranks the
/// detection and the defaults): the flag is kept and the question is
/// never re-asked, whatever the detection found (regression: a
/// confident detection used to overwrite the flag and an ambiguous one
/// re-asked the question).
#[test]
fn language_choice_flag_wins_over_detection() {
let detections = [
Detection::Single(Tid::Rust),
Detection::Ambiguous(vec![Tid::Rust, Tid::Python]),
Detection::Empty,
];
for detection in &detections {
for decides in [false, true] {
assert_eq!(
language_choice(Some("python"), detection, decides),
LanguageChoice::Flag,
"detection {detection:?}, decides {decides}"
);
}
}
}
/// Without the flag the detection behaves exactly as before: it decides
/// for a confident detection (packaging the detected directory), asks
/// with a preselection for a skeleton run, lists the candidates first
/// when ambiguous, and falls back to the plain menu otherwise.
#[test]
fn language_choice_without_flag_follows_detection() {
assert_eq!(
language_choice(None, &Detection::Single(Tid::Rust), true),
LanguageChoice::Detected(Tid::Rust)
);
// Skeleton run: ask, preselecting the detected ecosystem.
assert_eq!(
language_choice(None, &Detection::Single(Tid::Rust), false),
LanguageChoice::Ask(Some(Tid::Rust))
);
assert_eq!(
language_choice(
None,
&Detection::Ambiguous(vec![Tid::Go, Tid::Python]),
true
),
LanguageChoice::Ambiguous(vec![Tid::Go, Tid::Python])
);
// Nothing detected: plain menu, the empty template preselected.
assert_eq!(
language_choice(None, &Detection::Empty, false),
LanguageChoice::Ask(None)
);
}
#[test]
fn license_menu_and_defaults() {
let menu = license_menu();
@@ -1301,6 +1479,78 @@ mod tests {
assert!(required_answer("x")("ok").is_ok());
}
/// A probed upstream-version default must clear the same
/// [`options::validate_upstream_version`] bar that `resolve` applies: a
/// raw probe carrying a Debian revision or a `v` prefix fails it, so the
/// wizard withholds it instead of offering it for blind Enter-acceptance
/// and crashing late in `resolve` (regression).
#[test]
fn probed_version_defaults_fail_upstream_validation() {
assert!(options::validate_upstream_version("1.0-2", 1).is_err());
assert!(options::validate_upstream_version("v1.0", 1).is_err());
assert!(options::validate_upstream_version("1.0.0", 1).is_ok());
let validate = |version: &str| options::validate_upstream_version(version, 1);
// Invalid probes are not offered as the default...
assert_eq!(offered_default("1.0-2", &validate), "");
assert_eq!(offered_default("v1.0", &validate), "");
// ...clean probes keep theirs, and no default stays none.
assert_eq!(offered_default("1.0.0", &validate), "1.0.0");
assert_eq!(offered_default("", &validate), "");
}
/// [`accept_answer`] routes typed answers and Enter-taken defaults
/// through the same validation: valid typed input accepted verbatim,
/// invalid typed input rejected (re-ask), an empty answer taking a valid
/// default, and an empty answer with an invalid or missing default
/// rejected like any other invalid answer.
#[test]
fn accept_answer_validates_typed_answers_and_defaults() {
let validate = |answer: &str| {
if answer == "ok" {
Ok(())
} else {
Err(format!("not ok: {answer}"))
}
};
assert_eq!(accept_answer("ok", "ok", &validate), Ok("ok".to_string()));
assert!(accept_answer("bad", "ok", &validate).is_err());
// An empty answer takes the default, which must itself validate: an
// invalid (or missing) default is rejected, not taken.
assert_eq!(accept_answer("", "ok", &validate), Ok("ok".to_string()));
assert!(accept_answer("", "bad", &validate).is_err());
assert!(accept_answer("", "", &validate).is_err());
// The upstream-version scenario end to end: Enter on an invalid
// probed default (offered as none) fails instead of taking it
// verbatim, while a valid default is taken.
let version = |v: &str| options::validate_upstream_version(v, 1);
assert!(accept_answer("", "1.0-2", &version).is_err());
assert_eq!(
accept_answer("", "1.0.0", &version),
Ok("1.0.0".to_string())
);
}
/// The git-derived maintainer default goes through
/// [`options::parse_maintainer`] before it is offered: an empty git
/// email yields a `Name <>` default that parses to an error and is
/// ignored, instead of being accepted verbatim and only failing late in
/// `resolve` (regression).
#[test]
fn maintainer_default_validated_by_parse_maintainer() {
assert!(
options::parse_maintainer("Jane Doe <jane@example.com>").is_ok(),
"a well-formed identity is offered as the default"
);
// Empty git user.email (or DEBEMAIL): `Name <>` must not parse.
assert!(options::parse_maintainer("Jane Doe <>").is_err());
// An empty git user.name: `<>` (or `<email>`) must not parse either.
assert!(options::parse_maintainer(" <jane@example.com>").is_err());
}
#[test]
fn select_labels_carry_their_own_separator() {
// prompt::select renders `> <label><answer>` verbatim; a label
@@ -1313,4 +1563,92 @@ mod tests {
);
}
}
/// The probe helper is the same code path for the initial pass and the
/// re-probe: pointed at directory A it reports A's project, pointed at
/// directory B it reports B's. So when the source-location answer
/// redirects the wizard from the detected directory to another one, the
/// refreshed defaults (upstream version, description, homepage, license)
/// describe the project actually being packaged — regression: they used
/// to keep coming from the original directory.
#[test]
fn detect_and_probe_reports_the_directory_it_is_given() {
use tempfile::tempdir;
let manifest = |name: &str, version: &str, license: &str| {
format!(
"[package]\nname = \"{name}\"\nversion = \"{version}\"\n\
edition = \"2021\"\ndescription = \"{name} does {name} \
things\"\nhomepage = \"https://{name}.example.com\"\n\
license = \"{license}\"\n"
)
};
let dir_a = tempdir().unwrap();
std::fs::write(
dir_a.path().join("Cargo.toml"),
manifest("alpha", "0.1.0", "MIT"),
)
.unwrap();
let dir_b = tempdir().unwrap();
std::fs::write(
dir_b.path().join("Cargo.toml"),
manifest("beta", "2.9.9", "GPL-3.0+"),
)
.unwrap();
// The initial pass over dir A.
let (detection_a, probe_a) = detect_and_probe(dir_a.path());
assert_eq!(detection_a, Detection::Single(Tid::Rust));
let probe_a = probe_a.expect("dir A is a rust project");
assert_eq!(probe_a.name.as_deref(), Some("alpha"));
assert_eq!(probe_a.version.as_deref(), Some("0.1.0"));
assert_eq!(probe_a.license.as_deref(), Some("MIT"));
// The user chose dir B instead: the refreshed probe comes from B,
// never from A.
let (detection_b, probe_b) = detect_and_probe(dir_b.path());
assert_eq!(detection_b, Detection::Single(Tid::Rust));
let probe_b = probe_b.expect("dir B is a rust project");
assert_eq!(probe_b.name.as_deref(), Some("beta"));
assert_eq!(probe_b.version.as_deref(), Some("2.9.9"));
assert_eq!(
probe_b.description.as_deref(),
Some("beta does beta things")
);
assert_eq!(
probe_b.homepage.as_deref(),
Some("https://beta.example.com")
);
assert_eq!(probe_b.license.as_deref(), Some("GPL-3.0+"));
// A directory without project markers probes to nothing (the
// questions fall back to their plain defaults).
let dir_c = tempdir().unwrap();
let (detection_c, probe_c) = detect_and_probe(dir_c.path());
assert_eq!(detection_c, Detection::Empty);
assert!(probe_c.is_none());
}
/// The re-probe skip predicate: the detected/cwd directory never
/// re-probes — lexically identical paths and spellings of the same
/// directory (`.`, a trailing slash) all compare equal, different
/// directories and non-existent paths do not.
#[test]
fn same_directory_compares_spellings_of_one_directory() {
use tempfile::tempdir;
let dir = tempdir().unwrap();
let other = tempdir().unwrap();
assert!(same_directory(dir.path(), dir.path()));
// `.` names the current directory.
assert!(same_directory(
std::path::Path::new("."),
&std::env::current_dir().unwrap()
));
assert!(!same_directory(dir.path(), other.path()));
// A path that cannot be canonicalized only ever equals itself.
assert!(!same_directory(dir.path(), &dir.path().join("missing")));
}
}
+174 -18
View File
@@ -387,30 +387,44 @@ async fn get(
debug!("Fetching sources from: {}", url);
let response = match crate::distro_info::http_client().get(&url).send().await {
Ok(resp) => resp,
let compressed_data = match fetch_index_bytes(&url).await {
Ok(data) => data,
Err(e) => {
debug!("Failed to fetch {}: {}", url, e);
fetch_errors.push(format!("{suite}/{component}: {}", e));
debug!("Failed to fetch {url}: {e}");
fetch_errors.push(format!("{suite}/{component}: {e}"));
continue;
}
};
if !response.status().is_success() {
debug!("Failed to fetch {}: status {}", url, response.status());
fetch_errors.push(format!("{suite}/{component}: HTTP {}", response.status()));
continue;
}
let compressed_data = response.bytes().await?;
// The index must match the checksums listed in the signed Release
// file: this is what closes the 'substituted index with matching
// artifact checksums' man-in-the-middle attack
let suite_rel_path = format!("{component}/source/Sources.gz");
verified
.verify_file(&suite_rel_path, &compressed_data)
.map_err(release::VerifyError)?;
let compressed_data = match verified.verify_file(&suite_rel_path, &compressed_data) {
Ok(()) => compressed_data,
Err(verify_error) => {
// Busy mirrors and CDNs can serve an index generation
// slightly older or newer than the Release file fetched
// moments before. Pin the exact generation listed in the
// Release file via Debian's by-hash mechanism before
// failing.
match fetch_index_by_hash(&url, &verified, &suite_rel_path).await {
Ok(pinned) => {
debug!(
"index at '{url}' did not match the Release file: used the by-hash copy"
);
pinned
}
Err(by_hash_error) => {
debug!("by-hash fetch of '{}' failed: {}", url, by_hash_error);
return Err(release::VerifyError(format!(
"{verify_error}; the by-hash retry also failed: {by_hash_error}"
))
.into());
}
}
}
};
debug!(
"Downloaded Sources.gz for {}/{}/{}",
@@ -577,10 +591,102 @@ async fn get_flat_repo_series(base_url: &str) -> Result<String, Box<dyn Error>>
Err(format!("No Codename or Suite field in Release file at '{url}'").into())
}
/// Fetch an index pinned to the exact generation listed in the Release file
///
/// Uses Debian's by-hash mechanism, supported by the main archives (Debian,
/// Ubuntu, PPAs): the server serves the index generation matching the
/// Release file instead of whatever the mirror/CDN currently holds. The
/// returned content is verified again, so a by-hash-incapable mirror can
/// only cause a 404 here, never a silent mismatch.
async fn fetch_index_by_hash(
index_url: &str,
verified: &VerifiedRelease,
rel_path: &str,
) -> Result<Vec<u8>, Box<dyn Error>> {
let entry = verified.hash_for(rel_path).ok_or_else(|| {
format!("'{rel_path}' is not listed in the Release file: cannot fetch it by hash")
})?;
// By-hash URLs use the Release field names (MD5Sum, SHA1, SHA256, ...)
let algo = match entry.kind {
release::ChecksumKind::Md5 => "MD5Sum",
release::ChecksumKind::Sha1 => "SHA1",
release::ChecksumKind::Sha256 => "SHA256",
release::ChecksumKind::Sha512 => "SHA512",
};
// The by-hash layout replaces the file name with the hash reference:
// 'main/source/Sources.gz' is served at 'main/source/by-hash/SHA256/<digest>'
let (dir, _) = index_url
.rsplit_once('/')
.ok_or_else(|| format!("'{index_url}' has no parent directory"))?;
let url = format!("{dir}/by-hash/{algo}/{}", entry.hash);
let response = crate::distro_info::http_get_retried(&url).await?;
if !response.status().is_success() {
return Err(format!("HTTP {} for '{url}'", response.status()).into());
}
let data = response.bytes().await?.to_vec();
if data.is_empty() {
return Err(format!("empty body for '{url}'").into());
}
verified.verify_file(rel_path, &data)?;
Ok(data)
}
/// Fetch the compressed sources index at `url`
///
/// Transient transport failures and truncated bodies are retried; a
/// non-success HTTP status is a meaningful answer (e.g. a component that
/// does not exist in the suite) and is reported without retrying.
async fn fetch_index_bytes(url: &str) -> Result<Vec<u8>, String> {
const ATTEMPTS: u32 = 3;
let mut last_error = String::new();
for attempt in 1..=ATTEMPTS {
match crate::distro_info::http_get_retried(url).await {
Err(e) => {
last_error = e.to_string();
log::debug!("fetch of '{url}' failed (attempt {attempt}/{ATTEMPTS}): {last_error}");
tokio::time::sleep(std::time::Duration::from_millis(300 * u64::from(attempt)))
.await;
}
Ok(response) => {
if !response.status().is_success() {
return Err(format!("HTTP {}", response.status()));
}
match response.bytes().await {
Ok(bytes) if bytes.is_empty() => {
// CDNs occasionally answer 200 with an empty body
// under load: never a valid index, retry from scratch
last_error = "server returned an empty body".to_string();
log::debug!(
"empty body for '{url}' (attempt {attempt}/{ATTEMPTS}), retrying"
);
tokio::time::sleep(std::time::Duration::from_millis(
300 * u64::from(attempt),
))
.await;
}
Ok(bytes) => return Ok(bytes.to_vec()),
Err(e) => {
// Truncated or corrupted body: retry from scratch
last_error = e.to_string();
log::debug!(
"reading the body of '{url}' failed (attempt {attempt}/{ATTEMPTS}): {last_error}"
);
tokio::time::sleep(std::time::Duration::from_millis(
300 * u64::from(attempt),
))
.await;
}
}
}
}
}
Err(last_error)
}
/// Fetch the sources index of a flat repository
///
/// Flat repositories are free to serve any compressed variant of the index
/// (or an uncompressed one), so try the usual candidates in turn. When the
/// Flat repositories are free to serve any compressed variant of the index/// (or an uncompressed one), so try the usual candidates in turn. When the
/// repository published a Release file, each index candidate is
/// checksum-verified against it (failing on mismatch, since the artifact
/// hashes would come from the index itself).
@@ -592,7 +698,7 @@ async fn get_flat_repo_sources(
let mut errors = Vec::new();
for name in ["Sources.xz", "Sources.gz", "Sources"] {
let url = format!("{base}/{name}");
match crate::distro_info::http_client().get(&url).send().await {
match crate::distro_info::http_get_retried(&url).await {
Ok(response) if response.status().is_success() => {
let data = response.bytes().await?.to_vec();
@@ -837,6 +943,56 @@ pub async fn lookup(
mod tests {
use super::*;
/// Serve canned byte responses on a local port, one per connection (the
/// last response repeats), and return the base URL
///
/// The canned responses must use 'Connection: close' so the client opens
/// a fresh connection (and receives a fresh response) per request.
fn serve_responses(responses: Vec<Vec<u8>>) -> String {
use std::io::{Read, Write};
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap();
std::thread::spawn(move || {
for (served, stream) in listener.incoming().flatten().enumerate() {
let index = served.min(responses.len() - 1);
let mut stream = stream;
// Drain the request first: closing with unread inbound data
// would send a TCP RST and destroy the response in flight
let mut buf = [0u8; 4096];
loop {
match stream.read(&mut buf) {
Ok(0) => break,
Ok(n) if buf[..n].windows(4).any(|w| w == b"\r\n\r\n") => break,
Ok(_) => continue,
Err(_) => break,
}
}
let _ = stream.write_all(&responses[index]);
let _ = stream.flush();
}
});
format!("http://{addr}/Sources.gz")
}
/// A CDN answering 200 with an empty body (observed under load) must be
/// retried instead of failing the index checksum verification
#[tokio::test]
async fn fetch_index_bytes_retries_empty_body() {
let valid = b"Package: hello\nVersion: 1.0\n\n";
let empty_response =
b"HTTP/1.1 200 OK\r\nContent-Length: 0\r\nConnection: close\r\n\r\n".to_vec();
let mut valid_response = format!(
"HTTP/1.1 200 OK\r\nContent-Length: {}\r\nConnection: close\r\n\r\n",
valid.len()
)
.into_bytes();
valid_response.extend_from_slice(valid);
let url = serve_responses(vec![empty_response, valid_response]);
let data = fetch_index_bytes(&url).await.unwrap();
assert_eq!(data, valid);
}
#[tokio::test]
async fn test_check_launchpad_repo() {
// "hello" should exist on Launchpad for Ubuntu
+52 -14
View File
@@ -309,23 +309,49 @@ async fn download_file_checksum(
algo: crate::package_info::ChecksumAlgo,
target_dir: &Path,
progress: ProgressCallback<'_>,
) -> Result<(), Box<dyn Error>> {
// Archive mirrors and CDNs are busy enough that single attempts fail
// spuriously (dropped connections, truncated bodies, index generations
// momentarily out of sync): retry the whole download a few times before
// reporting the last failure.
const ATTEMPTS: u32 = 3;
let mut last_error: Box<dyn Error> = String::new().into();
for attempt in 1..=ATTEMPTS {
match download_file_checksum_once(url, checksum, algo, target_dir, progress).await {
Ok(()) => return Ok(()),
Err(e) => {
log::warn!("download of '{url}' failed (attempt {attempt}/{ATTEMPTS}): {e}");
last_error = e;
tokio::time::sleep(std::time::Duration::from_millis(500 * u64::from(attempt)))
.await;
}
}
}
Err(format!("downloading '{url}' failed after {ATTEMPTS} attempts: {last_error}").into())
}
/// One download attempt of [`download_file_checksum`], verifying the
/// content length and the expected checksum
async fn download_file_checksum_once(
url: &str,
checksum: &str,
algo: crate::package_info::ChecksumAlgo,
target_dir: &Path,
progress: ProgressCallback<'_>,
) -> Result<(), Box<dyn Error>> {
// Download with the shared client (connect timeout). Large orig tarballs
// can legitimately take longer than the client's default total timeout,
// so use a generous per-request timeout for streaming downloads
let response = crate::distro_info::http_client()
.get(url)
.timeout(std::time::Duration::from_secs(30 * 60))
.send()
.await?;
let response = crate::distro_info::http_get_retried_with_timeout(
url,
Some(std::time::Duration::from_secs(30 * 60)),
)
.await
.map_err(|e| Box::new(e) as Box<dyn Error>)?;
if !response.status().is_success() {
return Err(format!("Failed to download '{}' : {}", url, response.status()).into());
}
let total_size = response
.content_length()
.ok_or(format!("Failed to get content length from '{}'", url))?;
let mut index = 0;
let total_size = response.content_length();
// Target file: extract file name from URL
let filename = Path::new(url)
@@ -341,18 +367,30 @@ async fn download_file_checksum(
let mut stream = response.bytes_stream();
// Accumulate the downloaded bytes so we can compute the final digest with the
// correct algorithm once the download is complete.
let mut buffer: Vec<u8> = Vec::with_capacity(total_size as usize);
let mut buffer: Vec<u8> = Vec::with_capacity(total_size.unwrap_or(0) as usize);
while let Some(item) = stream.next().await {
let chunk = item?;
file.write_all(&chunk)?;
buffer.extend_from_slice(&chunk);
if let Some(cb) = progress {
index = min(index + chunk.len(), total_size as usize);
cb("", "Downloading...", index, total_size as usize);
if let (Some(cb), Some(total)) = (progress, total_size) {
let index = min(buffer.len(), total as usize);
cb("", "Downloading...", index, total as usize);
}
}
// A dropped connection can end the stream early: never hand a truncated
// file to the checksum check (its mismatch message would hide the cause)
if let Some(total) = total_size
&& buffer.len() != total as usize
{
return Err(format!(
"incomplete download from '{url}': got {} of {total} bytes",
buffer.len()
)
.into());
}
// Verify checksum using the algorithm specified for this file
let calculated_checksum = algo.hex_digest(&buffer);
if calculated_checksum != checksum {
+181 -28
View File
@@ -53,9 +53,10 @@ fn host_key_is_pinned(host: &str, fingerprint: &str) -> bool {
})
}
/// Per-host settings extracted from the SSH configuration files
/// (`/etc/ssh/ssh_config`, `~/.ssh/config`). Every option keeps the first
/// value obtained from the matching `Host` blocks, like OpenSSH.
/// Per-host settings extracted from the SSH configuration files, read in
/// the OpenSSH order (`~/.ssh/config`, then `/etc/ssh/ssh_config`). Every
/// option keeps the first value obtained from the matching `Host` blocks,
/// like OpenSSH: user settings override system ones.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct SshConfig {
/// `HostName`: real host behind a configured alias.
@@ -68,14 +69,19 @@ pub struct SshConfig {
pub identity_files: Vec<PathBuf>,
}
/// Look up `host` in the system and user SSH configuration files. Missing
/// files yield an empty configuration.
/// Look up `host` in the user and system SSH configuration files, in
/// OpenSSH's read order: `~/.ssh/config` first, then
/// `/etc/ssh/ssh_config` (command line > user > system, first obtained
/// value wins). Missing files yield an empty configuration.
pub fn lookup_ssh_config(host: &str) -> SshConfig {
let mut config = SshConfig::default();
let mut files = vec![PathBuf::from("/etc/ssh/ssh_config")];
// User file first: with first-obtained-value-wins semantics below, this
// makes user settings override the system ones, like OpenSSH
let mut files = Vec::new();
if let Some(home) = std::env::var_os("HOME") {
files.push(Path::new(&home).join(".ssh/config"));
}
files.push(PathBuf::from("/etc/ssh/ssh_config"));
for file in files {
if let Ok(content) = fs::read_to_string(&file) {
apply_config_file(&mut config, &content, host);
@@ -84,8 +90,13 @@ pub fn lookup_ssh_config(host: &str) -> SshConfig {
config
}
/// Fold one configuration file's matching `Host` blocks into `config`,
/// first obtained value wins per option.
/// Fold one configuration file's `Host` blocks matching `host` into
/// `config`, first obtained value wins per option. A block applies iff at
/// least one positive pattern matches and no negated (`!`-prefixed)
/// pattern matches — the OpenSSH `Host` rule. `Include` and `Match` are
/// not supported: a `Match` block's criteria are not evaluated, and it
/// conservatively stops attributing the following lines to the previous
/// `Host` block so its options are ignored rather than mis-applied.
fn apply_config_file(config: &mut SshConfig, content: &str, host: &str) {
let mut matching = false;
@@ -101,9 +112,15 @@ fn apply_config_file(config: &mut SshConfig, content: &str, host: &str) {
};
if keyword.eq_ignore_ascii_case("host") {
matching = rest
.split_whitespace()
.any(|pattern| match_pattern(host, pattern));
matching = match_pattern_list(host, rest);
continue;
}
if keyword.eq_ignore_ascii_case("match") {
// A conditional block whose criteria we do not evaluate: stop
// attributing the following options to the previous `Host`
// block, so they are ignored instead of mis-applied
matching = false;
continue;
}
@@ -120,7 +137,7 @@ fn apply_config_file(config: &mut SshConfig, content: &str, host: &str) {
config.port = Some(port);
}
} else if keyword.eq_ignore_ascii_case("identityfile") {
let path = PathBuf::from(rest);
let path = expand_tilde(rest);
if !config.identity_files.contains(&path) {
config.identity_files.push(path);
}
@@ -128,17 +145,53 @@ fn apply_config_file(config: &mut SshConfig, content: &str, host: &str) {
}
}
/// One `Host` block pattern against a host name: exact match, `*` (any
/// run), `?` (one character) and `!pattern` negation (a matching negated
/// pattern excludes the host from the block).
fn match_pattern(host: &str, pattern: &str) -> bool {
if let Some(negated) = pattern.strip_prefix('!') {
!match_pattern(host, negated)
/// The current user's home directory: the `directories` crate first (which
/// consults `$HOME` on Unix), with a direct `$HOME` fallback.
fn current_home() -> Option<PathBuf> {
directories::UserDirs::new()
.map(|dirs| dirs.home_dir().to_path_buf())
.or_else(|| std::env::var_os("HOME").map(PathBuf::from))
}
/// Expand a leading `~` or `~/` in `path` against `home`: the near-universal
/// `IdentityFile ~/.ssh/key` spelling. Only a leading tilde is handled —
/// `~user/...` (another user's home) and tildes appearing anywhere else are
/// kept verbatim — and without a known home directory the path is returned
/// unchanged.
fn expand_tilde_with(path: &str, home: Option<&Path>) -> PathBuf {
let Some(home) = home else {
return PathBuf::from(path);
};
if path == "~" {
home.to_path_buf()
} else if let Some(rest) = path.strip_prefix("~/") {
home.join(rest)
} else {
wildmatch(host, pattern)
PathBuf::from(path)
}
}
/// [`expand_tilde_with`] against the current user's home directory
fn expand_tilde(path: &str) -> PathBuf {
expand_tilde_with(path, current_home().as_deref())
}
/// One `Host` block pattern list against a host name, with the OpenSSH
/// rule: the block applies iff at least one positive pattern matches AND
/// no negated pattern matches — a matching `!pattern` excludes the host
/// from the whole block, regardless of the positive patterns.
fn match_pattern_list(host: &str, patterns: &str) -> bool {
let mut any_positive = false;
let mut any_negated = false;
for pattern in patterns.split_whitespace() {
match pattern.strip_prefix('!') {
Some(negated) => any_negated |= wildmatch(host, negated),
None => any_positive |= wildmatch(host, pattern),
}
}
any_positive && !any_negated
}
/// `*`/`?` glob matching without allocation, enough for `Host` patterns
fn wildmatch(text: &str, pattern: &str) -> bool {
let text: Vec<char> = text.chars().collect();
@@ -402,6 +455,16 @@ pub fn upload_file(
bar.inc(n as u64);
}
// Close explicitly: quota-exceeded and similar failures only surface in
// the final ACKs and the close handshake, and the `Drop` impl of
// `ssh2::File` discards that error ("too late to recover"), recording a
// truncated remote file as a successful upload. `ssh2::File::write` is
// unbuffered (`Write::flush` is a documented no-op) and `close`
// finalizes the pending writes server-side, so no flush is needed.
remote_file
.close()
.map_err(|e| format!("failed to close remote file '{remote}' after upload: {e}"))?;
Ok(())
}
@@ -526,16 +589,80 @@ mod tests {
));
}
/// Negated `Host` patterns exclude the host from the whole block (the
/// OpenSSH rule): a block applies iff at least one positive pattern
/// matches AND no negated pattern matches.
#[test]
fn pattern_negation_excludes_host() {
assert!(!match_pattern("ppa.launchpad.net", "!*.launchpad.net"));
// A negated pattern only excludes; the other patterns of the block
// decide separately
assert!(match_pattern("example.com", "!*.launchpad.net example.com"));
assert!(!match_pattern(
// The classic catch-all-with-exception: `*` alone would match, but
// the negated pattern must exclude the host from the whole block
let config = apply_config_file_all(
"\
Host * !*.launchpad.net
User fallback
",
"ppa.launchpad.net",
);
assert_eq!(config, SshConfig::default());
// A negation with no positive pattern never matches either
assert!(!match_pattern_list("ppa.launchpad.net", "!*.launchpad.net"));
// A matching negation excludes despite a positive match...
assert!(!match_pattern_list(
"ppa.launchpad.net",
"example.com !*.launchpad.net"
));
// ...while an unrelated negation leaves the positive match intact
assert!(match_pattern_list("ppa.launchpad.net", "* !*.example.com"));
}
/// OpenSSH's precedence: the user file is read before the system file
/// and the first value obtained wins, so the second (system) pass must
/// not override values the first (user) pass already obtained — while
/// options the user file leaves unset are still taken from the system
/// file.
#[test]
fn user_config_overrides_system() {
let user = "\
Host ppa.launchpad.net
User myuser
Port 2222
";
let system = "\
Host *
User login
Port 22
HostName system.example.com
";
let mut config = SshConfig::default();
apply_config_file(&mut config, user, "ppa.launchpad.net"); // ~/.ssh/config
apply_config_file(&mut config, system, "ppa.launchpad.net"); // /etc/ssh/ssh_config
assert_eq!(config.user.as_deref(), Some("myuser"));
assert_eq!(config.port, Some(2222));
assert_eq!(config.host_name.as_deref(), Some("system.example.com"));
}
/// `Match` blocks are not evaluated: their options must be ignored, not
/// mis-attributed to the preceding `Host` block through a stale
/// matching flag.
#[test]
fn match_block_is_ignored_conservatively() {
let config = apply_config_file_all(
"\
Host ppa.launchpad.net
User myuser
Match final all
Port 2223
IdentityFile ~/.ssh/matched_key
",
"ppa.launchpad.net",
);
assert_eq!(config.user.as_deref(), Some("myuser"));
assert_eq!(config.port, None);
assert!(config.identity_files.is_empty());
}
#[test]
@@ -557,17 +684,43 @@ Host *
);
assert_eq!(config.user.as_deref(), Some("myuser"));
// A leading tilde is expanded to the home directory, built from the
// same lookup the helper uses so no specific username is assumed
let home = current_home().expect("tests require a home directory");
assert_eq!(
config.identity_files,
vec![
PathBuf::from("~/.ssh/lp_key"),
PathBuf::from("~/.ssh/other_key")
]
vec![home.join(".ssh/lp_key"), home.join(".ssh/other_key")]
);
assert_eq!(config.host_name, None);
assert_eq!(config.port, None);
}
/// Only a leading `~`/`~/` expands to the home directory; `~user/...`
/// and non-tilde paths are kept verbatim, and nothing is expanded
/// without a known home directory
#[test]
fn expand_tilde_handles_leading_tilde_only() {
let home = Some(Path::new("/home/testuser"));
assert_eq!(
expand_tilde_with("~/x", home),
PathBuf::from("/home/testuser/x")
);
assert_eq!(
expand_tilde_with("~", home),
PathBuf::from("/home/testuser")
);
assert_eq!(
expand_tilde_with("~other/x", home),
PathBuf::from("~other/x")
);
assert_eq!(
expand_tilde_with("relative", home),
PathBuf::from("relative")
);
assert_eq!(expand_tilde_with("/abs", home), PathBuf::from("/abs"));
assert_eq!(expand_tilde_with("~/x", None), PathBuf::from("~/x"));
}
#[test]
fn config_wildcard_and_question_marks_match() {
let config = apply_config_file_all(
+486
View File
@@ -0,0 +1,486 @@
//! Test-run support: quiet console, per-test log files and an end-of-run
//! failure matrix.
//!
//! In test binaries (registered from `lib.rs`) and initialized before `main`
//! through an `.init_array` entry so every test gets the same treatment, with
//! or without `#[test_log::test]`:
//!
//! - all `log` output is written to a per-test file under
//! `target/pkh-test-logs/` instead of the terminal, so concurrent tests
//! never interleave their logs;
//! - subprocesses launched through a [`crate::context::Context`] (the whole
//! build pipeline: apt, dpkg, make, compilers, ...) are captured line by
//! line into the same per-test file instead of inheriting the terminal,
//! which used to garble the `cargo test` output beyond readability;
//! - [`run_logged`] does the same for direct `std::process::Command` spawns
//! from test code (e.g. the differential `dpkg-buildpackage` runs);
//! - a panic hook records test failures, and an `atexit` callback prints a
//! failure matrix (test name, panic message, log file path) right after
//! the libtest summary.
//!
//! Outside of test builds everything is an inert passthrough stub: the
//! production CLI behaves exactly as before.
//!
//! Attribution relies on libtest naming each test's thread after the test —
//! true for the default parallel runner. With `--test-threads=1` tests run
//! on the anonymous main thread and share `_uncategorized.log` instead; the
//! failure matrix then points at that file, and the panic location still
//! identifies the failing test.
//!
//! The panic hook records every panic of the process: the suite currently
//! has no `#[should_panic]` tests, so no filtering is needed; if one is
//! added, exclude it in `record_failure`.
/// Production builds get passthrough stubs: no logger hijacking, no log
/// directory, commands inherit the terminal as always.
#[cfg(not(test))]
#[allow(dead_code)]
mod imp {
pub(crate) fn active() -> bool {
false
}
pub(crate) fn subprocess_sink() -> Option<std::sync::Arc<dyn crate::context::LineSink>> {
None
}
pub(crate) fn wrap_driver(
driver: Box<dyn crate::context::ContextDriver + Send + Sync>,
) -> Box<dyn crate::context::ContextDriver + Send + Sync> {
driver
}
pub(crate) fn run_logged(
cmd: &mut std::process::Command,
) -> std::io::Result<std::process::ExitStatus> {
cmd.status()
}
pub(crate) fn suppress_failure_recording() -> SuppressFailuresStub {
SuppressFailuresStub
}
/// Type returned by the production stub of
/// [`suppress_failure_recording`]
pub(crate) struct SuppressFailuresStub;
}
pub(crate) use imp::*;
#[cfg(test)]
mod imp {
use std::collections::HashMap;
use std::fmt::Write as _;
use std::fs::{self, File};
use std::io::{self, Write as _};
use std::panic::PanicHookInfo;
use std::path::{Path, PathBuf};
use std::process::{Command, ExitStatus};
use std::sync::{Arc, Mutex, OnceLock};
use crate::context::{ContextDriver, LineSink, Stream};
/// Directory receiving one log file per test
fn log_dir() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR")).join("target/pkh-test-logs")
}
/// Log file of the current test thread, if attributable to a test
///
/// Returns `None` for the anonymous `main` thread (`--test-threads=1`)
/// and for helper threads spawned by libraries: their output would be
/// misattributed, so it goes to the shared `_uncategorized.log`.
fn current_test_log_path() -> Option<PathBuf> {
let thread = std::thread::current();
let name = thread.name()?;
if name == "main" {
return None;
}
Some(log_dir().join(format!("{}.log", sanitize_test_name(name))))
}
/// File name-safe version of a test path (`a::b` → `a__b`)
fn sanitize_test_name(name: &str) -> String {
name.chars()
.map(|c| {
if c.is_ascii_alphanumeric() || c == '-' || c == '_' {
c
} else {
'_'
}
})
.collect()
}
/// Log file used when output cannot be attributed to one test
fn uncategorized_log_path() -> PathBuf {
log_dir().join("_uncategorized.log")
}
/// Opened per-test log files, shared so that subprocess-capture threads
/// can append to the file of the test that started the command
fn files() -> &'static Mutex<HashMap<PathBuf, Arc<Mutex<File>>>> {
static FILES: OnceLock<Mutex<HashMap<PathBuf, Arc<Mutex<File>>>>> = OnceLock::new();
FILES.get_or_init(|| Mutex::new(HashMap::new()))
}
/// Append one already-formatted line to the per-test log file at `path`,
/// creating it (with a header) on first use
fn append_line(path: &Path, line: &str) {
let file = {
let mut files = files().lock().unwrap_or_else(|e| e.into_inner());
files
.entry(path.to_path_buf())
.or_insert_with(|| {
Arc::new(Mutex::new(File::create(path).unwrap_or_else(|e| {
panic!("cannot open test log file {}: {e}", path.display())
})))
})
.clone()
};
let mut file = file.lock().unwrap_or_else(|e| e.into_inner());
let _ = writeln!(file, "{line}");
let _ = file.flush();
}
/// Whether a test is running and per-test logging is set up
pub(crate) fn active() -> bool {
ACTIVE.load(std::sync::atomic::Ordering::Relaxed)
}
static ACTIVE: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
// Runs before `main` of the test binary: installs the logger, the panic
// hook and the end-of-run matrix printer before any test starts.
#[used]
#[unsafe(link_section = ".init_array")]
static INIT: extern "C" fn() = init;
extern "C" fn init() {
ACTIVE.store(true, std::sync::atomic::Ordering::Relaxed);
// Do not prune the log directory here: `cargo test` may run several
// test binaries in sequence and later ones must not destroy the
// logs of earlier ones. Per-test files are truncated on first write,
// so the current run's logs are always current.
let _ = fs::create_dir_all(log_dir());
if log::set_boxed_logger(Box::new(TestLogger)).is_ok() {
log::set_max_level(log_filter_from_env());
}
let previous = std::panic::take_hook();
std::panic::set_hook(Box::new(move |info| {
record_failure(info);
previous(info);
}));
// SAFETY: `print_failure_matrix` is a plain function pointer with no
// argument; registering it as an exit handler is infallible.
unsafe {
libc::atexit(print_failure_matrix);
}
}
/// Log level from `RUST_LOG`, defaulting to `info` like the CLI
fn log_filter_from_env() -> log::LevelFilter {
match std::env::var("RUST_LOG").as_deref() {
Ok("trace") => log::LevelFilter::Trace,
Ok("debug") => log::LevelFilter::Debug,
Ok("warn") => log::LevelFilter::Warn,
Ok("error") => log::LevelFilter::Error,
Ok("off") => log::LevelFilter::Off,
_ => log::LevelFilter::Info,
}
}
/// Logger writing every record into the current test's log file
struct TestLogger;
impl log::Log for TestLogger {
fn enabled(&self, metadata: &log::Metadata) -> bool {
metadata.level() <= log::max_level()
}
fn log(&self, record: &log::Record) {
if !self.enabled(record.metadata()) {
return;
}
let timestamp = chrono::Utc::now().format("%H:%M:%S%.3f");
let line = format!(
"{timestamp} {:<5} {}: {}",
record.level(),
record.target(),
record.args()
);
append_line(
&current_test_log_path().unwrap_or_else(uncategorized_log_path),
&line,
);
}
fn flush(&self) {}
}
/// One recorded test panic
struct Failure {
/// Name of the panicking thread: the test path for parallel runs
test: String,
/// Source location of the panic
location: String,
/// Panic message
message: String,
/// Per-test log file
log: PathBuf,
}
static FAILURES: Mutex<Vec<Failure>> = Mutex::new(Vec::new());
/// While alive on the current thread, panics are not recorded as test
/// failures
///
/// For tests that panic on purpose (e.g. a cleanup hook whose panic is
/// caught and part of the behavior under test): without this they would
/// show up as phantom entries in the end-of-run failure matrix.
pub(crate) fn suppress_failure_recording() -> SuppressFailures {
SUPPRESSED.with(|count| count.set(count.get() + 1));
SuppressFailures
}
thread_local! {
static SUPPRESSED: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
pub(crate) struct SuppressFailures;
impl Drop for SuppressFailures {
fn drop(&mut self) {
SUPPRESSED.with(|count| count.set(count.get().saturating_sub(1)));
}
}
fn record_failure(info: &PanicHookInfo<'_>) {
if SUPPRESSED.with(|count| count.get() > 0) {
return;
}
let thread = std::thread::current();
let test = thread.name().unwrap_or("<unnamed>").to_string();
let location = info
.location()
.map(|l| format!("{}:{}:{}", l.file(), l.line(), l.column()))
.unwrap_or_default();
let message = payload_message(info.payload());
let log = current_test_log_path().unwrap_or_else(uncategorized_log_path);
append_line(&log, &format!("PANIC: {message}"));
FAILURES
.lock()
.unwrap_or_else(|e| e.into_inner())
.push(Failure {
test,
location,
message,
log,
});
}
fn payload_message(payload: &(dyn std::any::Any + Send)) -> String {
if let Some(s) = payload.downcast_ref::<&str>() {
(*s).to_string()
} else if let Some(s) = payload.downcast_ref::<String>() {
s.clone()
} else {
"panic with non-string payload".to_string()
}
}
/// Print the failure matrix after the libtest summary (registered via
/// `atexit`, so it always runs last)
extern "C" fn print_failure_matrix() {
let failures = FAILURES.lock().unwrap_or_else(|e| e.into_inner());
if failures.is_empty() {
return;
}
let mut out = String::new();
let _ = writeln!(
out,
"\n═══ pkh test failures: {} ═══ per-test logs in {} ═══",
failures.len(),
log_dir().display()
);
for failure in failures.iter() {
let _ = writeln!(out, " • {}", failure.test);
if !failure.location.is_empty() {
let _ = writeln!(out, " at {}", failure.location);
}
let _ = writeln!(out, " panic {}", failure.message);
let _ = writeln!(out, " log {}", failure.log.display());
}
let mut stderr = io::stderr().lock();
let _ = stderr.write_all(out.as_bytes());
let _ = stderr.flush();
}
/// Line sink forwarding subprocess output into a test's log file
struct TestFileSink {
path: PathBuf,
}
impl LineSink for TestFileSink {
fn line(&self, stream: Stream, line: &str) {
let label = match stream {
Stream::Stdout => "out",
Stream::Stderr => "err",
};
append_line(&self.path, &format!("[{label}] {line}"));
}
}
/// Capture sink for subprocesses launched from the current test
///
/// Returns `None` outside of test runs, letting callers keep their
/// normal inherit-or-UI-sink behavior.
pub(crate) fn subprocess_sink() -> Option<Arc<dyn LineSink>> {
if !active() {
return None;
}
Some(Arc::new(TestFileSink {
path: current_test_log_path().unwrap_or_else(uncategorized_log_path),
}))
}
/// Wrap a context driver so that plain (uncaptured) `run`/`run_output`
/// commands still have their output logged in test runs instead of
/// inheriting the terminal
pub(crate) fn wrap_driver(
driver: Box<dyn ContextDriver + Send + Sync>,
) -> Box<dyn ContextDriver + Send + Sync> {
if !active() {
return driver;
}
Box::new(CapturingDriver { inner: driver })
}
struct CapturingDriver {
inner: Box<dyn ContextDriver + Send + Sync>,
}
impl ContextDriver for CapturingDriver {
fn run(
&self,
program: &str,
args: &[String],
env: &[(String, String)],
cwd: Option<&str>,
) -> io::Result<ExitStatus> {
match subprocess_sink() {
Some(sink) => self.inner.run_captured(program, args, env, cwd, sink),
None => self.inner.run(program, args, env, cwd),
}
}
fn run_captured(
&self,
program: &str,
args: &[String],
env: &[(String, String)],
cwd: Option<&str>,
sink: Arc<dyn LineSink>,
) -> io::Result<ExitStatus> {
self.inner.run_captured(program, args, env, cwd, sink)
}
fn run_output(
&self,
program: &str,
args: &[String],
env: &[(String, String)],
cwd: Option<&str>,
) -> io::Result<std::process::Output> {
let sink = subprocess_sink();
// Commands whose stdout is parsed still need the real output, so
// delegate to run_output and additionally forward the lines to
// the per-test log
let output = self.inner.run_output(program, args, env, cwd)?;
if let Some(sink) = sink {
for line in String::from_utf8_lossy(&output.stdout).lines() {
sink.line(Stream::Stdout, line);
}
for line in String::from_utf8_lossy(&output.stderr).lines() {
sink.line(Stream::Stderr, line);
}
}
Ok(output)
}
fn ensure_available(&self, src: &Path, dest_root: &str) -> io::Result<PathBuf> {
self.inner.ensure_available(src, dest_root)
}
fn retrieve_path(&self, src: &Path, dest: &Path) -> io::Result<()> {
self.inner.retrieve_path(src, dest)
}
fn list_files(&self, path: &Path) -> io::Result<Vec<PathBuf>> {
self.inner.list_files(path)
}
fn create_temp_dir(&self) -> io::Result<String> {
self.inner.create_temp_dir()
}
fn copy_path(&self, src: &Path, dest: &Path) -> io::Result<()> {
self.inner.copy_path(src, dest)
}
fn read_file(&self, path: &Path) -> io::Result<String> {
self.inner.read_file(path)
}
fn write_file(&self, path: &Path, content: &str) -> io::Result<()> {
self.inner.write_file(path, content)
}
fn exists(&self, path: &Path) -> io::Result<bool> {
self.inner.exists(path)
}
fn cleanup(&self) -> io::Result<()> {
self.inner.cleanup()
}
}
/// Run a command from test code with its output captured into the
/// per-test log file (the console stays quiet; the output is available
/// on failure)
pub(crate) fn run_logged(cmd: &mut Command) -> io::Result<ExitStatus> {
if !active() {
return cmd.status();
}
let path = current_test_log_path().unwrap_or_else(uncategorized_log_path);
append_line(&path, &format!("[run] {}", display_command(cmd)));
append_line(&path, &format!("[run] cwd: {}", display_cwd(cmd)));
let output = cmd.stdin(std::process::Stdio::null()).output()?;
for line in String::from_utf8_lossy(&output.stdout).lines() {
append_line(&path, &format!("[out] {line}"));
}
for line in String::from_utf8_lossy(&output.stderr).lines() {
append_line(&path, &format!("[err] {line}"));
}
append_line(&path, &format!("[run] status: {}", output.status));
Ok(output.status)
}
fn display_command(cmd: &Command) -> String {
let mut parts = vec![cmd.get_program().to_string_lossy().to_string()];
parts.extend(cmd.get_args().map(|a| a.to_string_lossy().to_string()));
parts.join(" ")
}
fn display_cwd(cmd: &Command) -> String {
cmd.get_current_dir()
.map(|d| d.display().to_string())
.unwrap_or_else(|| "<current>".to_string())
}
}
+12
View File
@@ -9,6 +9,8 @@ pub mod logfmt;
/// yes/no confirmation
pub mod prompt;
#[cfg(test)]
use indicatif::ProgressDrawTarget;
use indicatif::{MultiProgress, ProgressBar, ProgressStyle};
use std::path::Path;
use std::time::Duration;
@@ -38,6 +40,16 @@ pub(crate) fn spinner_style() -> ProgressStyle {
.unwrap()
}
/// Progress draw target for a [`MultiProgress`]: hidden in test runs
///
/// Steady-tick spinners redraw from a background thread straight to the real
/// stderr, bypassing both the test harness capture and the per-test log
/// files; in tests there is no terminal to animate anyway.
#[cfg(test)]
pub(crate) fn progress_draw_target() -> ProgressDrawTarget {
ProgressDrawTarget::hidden()
}
/// Style of a sized transfer: prefix on the first line, the bar on its own
/// indented line below so long prefixes cannot push it out of the terminal
pub(crate) fn transfer_style() -> ProgressStyle {