Files
vhaudiquet 9acf650fc9 deb,config: the staged-tree budget replaces the size cap
The size cap was the wrong shape: an --incremental user asked for a
carried tree, and the invariant that matters is which trees survive —
the newest successful one (the next run's automatic fork base) and the
newest one (what a failed run is re-forked from explicitly). The cap
is gone, with the tree measurement and manifest field it needed; the
retention planner now works on the records' outcomes alone and fills a
budget: staged_trees staged trees per target, the newer ones first,
the pair always among them. When the budget cannot hold the pair (one
staged tree configured, the newest build failed and an older
successful tree present) only one of the two is successful and that
one is kept; with no successful tree at all, the most recent one is.

The budget is a setting of the nested config file:

    deb:
      incremental:
        staged_trees: 2

(default 2; 0 clamps to 1 with a warning — the newest successful tree
always survives). The fork-base selection no longer truncates the
candidate list to the budget before ranking it: with a budget of one
and a failed newest tree, that truncation hid the successful record
and forked the failed one instead.
2026-10-03 23:43:10 +02:00

1909 lines
70 KiB
Rust

//! Cleanup of residual pkh build artifacts and caches.
//!
//! `pkh` leaves behind several kinds of artifacts while operating:
//!
//! - **Residual build/chroot directories** under the system temp directory,
//! named `pkh-<timestamp>` (or `pkh-<timestamp>-<attempt>` on collision).
//! They are created by the local and ephemeral build contexts and are
//! *intentionally kept* when a build fails so they can be inspected. They
//! may contain root-owned device nodes (created via `mknod`), a
//! bind-mounted `/proc` and overlay filesystems, so they require careful
//! unmounting before they can be removed.
//! - **Build records** under `/var/tmp/pkh/sessions/<slug>/<record-id>/`
//! (see [`crate::deb::session`]): one record per build (metadata and
//! log for every run, the staged tree of `--incremental` ones); the
//! default policy removes the ones untouched for longer than the
//! record retention window plus the corrupt leftovers, `--all`
//! removes every record — recent builds and staged trees included,
//! which is where the space is. A record a live build holds the lock
//! of is never prunable, whatever the policy (the newest and the
//! newest successful staged trees per target are kept by `pkh deb`,
//! two by default).
//! - **Shared apt download caches** under `/var/tmp/pkh/apt-cache/`
//! (see [`crate::deb::ephemeral::apt_cache_dir`]): the default policy
//! removes the ones untouched for longer than the retention window,
//! `--all` removes them all; the `.debs` they hold are cheap to
//! re-download.
//! - **Cached chroot tarballs** (`~/.cache/pkh/*-buildd.tar.xz`) and their
//! **stale download lockfiles** (`~/.cache/pkh/*.lock` untouched for longer
//! than [`LOCK_STALE_AFTER`]; younger lockfiles may belong to a concurrent
//! `pkh` run and are left alone).
//! - **The apt keyring cache directories** (`pkh-keyrings` and the per-uid
//! `pkh-keyrings-<uid>` under the system temp directory), used by
//! mmdebstrap runs.
//! - **Build logs** under `~/.cache/pkh/logs/` written by `pkh deb`:
//! `deb-<package>-<version>-<timestamp>.log` (binary builds),
//! `build-<package>-<version>-<timestamp>.log` (source builds) and the
//! pre-identity placeholder `pkh-<timestamp>.log`. By default only logs
//! beyond a small retention window (the newest [`KEEP_LOGS`] by embedded
//! timestamp are kept) are removed; pass [`PruneOptions::all`] to remove
//! them all.
//!
//! The [`prune()`] function discovers and removes all of the above. By
//! default it removes everything that is cheap to regenerate (residual
//! directories, keyrings, stale lockfiles, old build logs); pass
//! [`PruneOptions::all`] to also discard the cached chroot tarballs, all
//! build logs and the recent build records. [`PruneOptions::dry_run`]
//! reports the same set grouped by kind with sizes, without removing
//! anything.
use std::fs;
use std::path::{Path, PathBuf};
use std::process::Command;
use std::time::{Duration, SystemTime};
use chrono::NaiveDateTime;
use directories::ProjectDirs;
/// Options controlling the prune operation.
#[derive(Clone, Copy, Default)]
pub struct PruneOptions {
/// When true, do not remove anything; only report what would be removed.
pub dry_run: bool,
/// When true, also remove cached chroot tarballs (which are expensive to
/// re-download) and recent build records with their staged trees.
/// Without this, only artifacts beyond their retention windows are
/// removed.
pub all: bool,
/// When true, skip the size computation and the confirmation prompt:
/// remove immediately, reporting counts but no sizes. A dry run
/// still wins over this.
pub force: bool,
}
/// Summary of a prune run.
#[derive(Default)]
pub struct PruneReport {
/// Whether this was a dry run (nothing was actually removed).
pub dry_run: bool,
/// Paths that were removed, or that *would* be removed in dry-run mode.
pub removed: Vec<PathBuf>,
/// The removed paths grouped by artifact kind, with sizes. Sizes are
/// measured before removal, whatever the mode.
pub groups: Vec<PruneGroup>,
/// Total size across the groups.
pub total_bytes: u64,
/// Paths that could not be removed, with associated error messages.
pub failed: Vec<(PathBuf, String)>,
}
impl PruneReport {
/// Returns true if nothing was removed and nothing failed.
pub fn is_empty(&self) -> bool {
self.removed.is_empty() && self.failed.is_empty()
}
}
/// One artifact category in a [`PruneReport`]: the artifact kind, how
/// many entries of it were (or would be) removed, and their total size.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PruneGroup {
/// The artifact kind (singular, as discovered).
pub kind: &'static str,
/// How many entries of this kind were removed.
pub count: usize,
/// Total size of the entries in bytes.
pub bytes: u64,
}
impl PruneGroup {
/// The category's display label, pluralized for the count
/// ("1 stale lockfile", "2 stale lockfiles").
pub fn label(&self) -> String {
if self.count == 1 {
return self.kind.to_string();
}
match self.kind.strip_suffix('y') {
Some(stem) => format!("{stem}ies"),
None => format!("{}s", self.kind),
}
}
}
/// Human-readable size for the prune report ("512 B", "3.5 MiB",
/// "1.2 GiB", "1.3 TiB").
pub fn format_bytes(bytes: u64) -> String {
crate::utils::size::format_bytes(bytes)
}
/// Total size of every artifact in bytes, measured concurrently (the
/// walker of [`crate::utils::size`]).
fn size_artifacts(artifacts: &[&Artifact]) -> Vec<u64> {
let paths: Vec<&Path> = artifacts.iter().map(|a| a.path()).collect();
crate::utils::size::tree_sizes(&paths)
}
/// Number of newest build logs kept when `--all` is not passed.
pub(crate) const KEEP_LOGS: usize = 10;
/// Lockfiles younger than this are not pruned: a recently touched
/// `<tarball>.lock` may belong to a concurrent `pkh` run, whose chroot
/// tarball download uses the lockfile's mere existence as a mutual-exclusion
/// signal (see `deb::ephemeral`). Deleting such a lockfile mid-download would
/// let two builds corrupt the shared tarball cache. Only lockfiles untouched
/// for at least this long are considered stale.
const LOCK_STALE_AFTER: Duration = Duration::from_secs(24 * 60 * 60);
/// One discovered artifact that prune can act on.
#[derive(Debug, Clone)]
enum Artifact {
/// A residual build/chroot directory under the system temp dir.
TempDir(PathBuf),
/// The apt keyring cache directory (`pkh-keyrings*` under the system
/// temp dir).
KeyringDir(PathBuf),
/// A stale chroot tarball download lockfile.
LockFile(PathBuf),
/// A cached chroot tarball.
Tarball(PathBuf),
/// A build log file under `<cache>/logs`.
LogFile(PathBuf),
/// A build record root (manifest + log, and the staged tree of
/// `--incremental` runs), one per build.
Session(PathBuf),
/// A `pkh test` run directory (report artifacts).
TestRun(PathBuf),
/// A shared apt download cache of one target (see
/// [`crate::deb::ephemeral::apt_cache_dir`]).
AptCache(PathBuf),
/// An orphaned record lock file (`<record id>.lock` whose record
/// root is gone).
SessionLock(PathBuf),
}
impl Artifact {
fn path(&self) -> &Path {
match self {
Artifact::TempDir(p)
| Artifact::KeyringDir(p)
| Artifact::LockFile(p)
| Artifact::Tarball(p)
| Artifact::LogFile(p)
| Artifact::Session(p)
| Artifact::TestRun(p)
| Artifact::AptCache(p)
| Artifact::SessionLock(p) => p,
}
}
fn kind(&self) -> &'static str {
match self {
Artifact::TempDir(_) => "residual build directory",
Artifact::KeyringDir(_) => "apt keyring cache",
Artifact::LockFile(_) => "stale lockfile",
Artifact::Tarball(_) => "cached chroot tarball",
Artifact::LogFile(_) => "build log",
Artifact::Session(_) => "build record",
Artifact::TestRun(_) => "pkh test run",
Artifact::AptCache(_) => "shared apt download cache",
Artifact::SessionLock(_) => "stale record lockfile",
}
}
}
/// Discover the build records prune may remove under `sessions_dir` (the
/// record layout of [`crate::deb::session`]: one `<record-id>` directory
/// — with its `<record-id>.lock` beside it — per slug directory): with
/// `--all`, every record (recent builds and staged trees included);
/// otherwise only those untouched for longer than the retention window
/// plus the corrupt leftovers. A record a live build holds the lock of
/// is out of bounds either way.
///
/// Orphaned lock files (`<record id>.lock` whose record root is gone —
/// left behind by interrupted or discarded runs) are collected too,
/// unless a live build holds them (probed with a non-blocking flock).
/// The pre-record single-session layout kept its lock at the sessions
/// root and its manifest directly inside the slug directory; both are
/// collected as the leftovers they became.
fn discover_sessions(sessions_dir: &Path, all: bool) -> Vec<Artifact> {
let Ok(entries) = fs::read_dir(sessions_dir) else {
return Vec::new();
};
let mut artifacts = Vec::new();
for entry in entries.flatten() {
let path = entry.path();
let name = entry.file_name().to_string_lossy().into_owned();
// The old layout's `<slug>.lock` at the root: orphaned once no
// slug directory of that name exists.
if let Some(slug) = name.strip_suffix(".lock") {
if path.is_file() && !sessions_dir.join(slug).is_dir() {
collect_lock(&path, &mut artifacts);
}
continue;
}
let Some(slug_dir) = path.is_dir().then_some(path) else {
continue;
};
let Ok(records) = fs::read_dir(&slug_dir) else {
continue;
};
for record in records.flatten() {
let path = record.path();
let name = record.file_name().to_string_lossy().into_owned();
if let Some(record_id) = name.strip_suffix(".lock") {
if path.is_file() && !slug_dir.join(record_id).is_dir() {
collect_lock(&path, &mut artifacts);
}
} else if path.is_dir() {
if is_prunable_session(&path, all) {
artifacts.push(Artifact::Session(path));
}
} else if is_stale_path(&path, all) {
// A record root only ever holds directories; a plain
// file inside a slug directory is a leftover of the old
// flat layout (its manifest / host-tree snapshot).
artifacts.push(Artifact::Session(path));
}
}
}
artifacts
}
/// Collect an orphaned lock file as prunable. A lock a live build holds
/// is never collected, `--all` included: it is mutual exclusion for a
/// concurrent run, not garbage.
fn collect_lock(path: &Path, artifacts: &mut Vec<Artifact>) {
if lock_held(path) {
return;
}
artifacts.push(Artifact::SessionLock(path.to_path_buf()));
}
/// Whether a path is untouched for longer than the record retention
/// window (mtime-based; the fallback for leftovers with no manifest).
fn is_stale_path(path: &Path, all: bool) -> bool {
all || fs::metadata(path)
.and_then(|meta| meta.modified())
.is_ok_and(|mtime| {
chrono::Utc::now()
.signed_duration_since(chrono::DateTime::<chrono::Utc>::from(mtime))
.to_std()
.is_ok_and(|age| age >= crate::deb::session::SESSION_RETENTION)
})
}
/// Discover the `pkh test` run directories prune may remove under
/// `tests_dir`: with `--all`, every run; otherwise only those untouched
/// for longer than the sessions retention window.
fn discover_test_runs(tests_dir: &Path, all: bool) -> Vec<Artifact> {
let Ok(entries) = fs::read_dir(tests_dir) else {
return Vec::new();
};
entries
.flatten()
.map(|entry| entry.path())
.filter(|path| path.is_dir())
.filter(|path| is_stale_path(path, all))
.map(Artifact::TestRun)
.collect()
}
/// Discover the shared apt download caches prune may remove under
/// `apt_cache_dir` (`/var/tmp/pkh/apt-cache`, one `<series>-<arch>`
/// subdirectory per target): with `--all`, every one; otherwise only
/// those untouched for longer than the retention window. A cache in use
/// by a live build is fresh by construction — its `.debs` are being
/// read and written.
fn discover_apt_cache(apt_cache_dir: &Path, all: bool) -> Vec<Artifact> {
let Ok(entries) = fs::read_dir(apt_cache_dir) else {
return Vec::new();
};
entries
.flatten()
.map(|entry| entry.path())
.filter(|path| path.is_dir())
.filter(|path| is_stale_path(path, all))
.map(Artifact::AptCache)
.collect()
}
/// Whether a record root is prunable under the given policy: everything
/// with `--all` (recent builds and staged trees included — that is where
/// the space is), otherwise stale (untouched beyond the retention window)
/// or corrupt (no readable manifest) records. A record a live build holds
/// the lock of is never prunable, whatever the policy: pruning it would
/// wreck the running build. Live records record a fresh `last_used` when
/// they are created and are never stale; a record whose manifest is
/// missing is mid-creation rather than corrupt while its lock is held,
/// and is left alone.
fn is_prunable_session(path: &Path, all: bool) -> bool {
if lock_held(&record_lock_path(path)) {
return false;
}
if all {
return true;
}
let last_used = fs::read_to_string(path.join("manifest.json"))
.ok()
.and_then(|content| serde_json::from_str::<serde_json::Value>(&content).ok())
.and_then(|value| {
value
.get("last_used")
.and_then(|v| v.as_str())
.and_then(|s| chrono::DateTime::parse_from_rfc3339(s).ok())
});
match last_used {
Some(then) => chrono::Utc::now()
.signed_duration_since(chrono::DateTime::<chrono::Utc>::from(then))
.to_std()
.is_ok_and(|age| age >= crate::deb::session::SESSION_RETENTION),
// No readable manifest: a leftover (the lock is not held, per
// the check above).
None => true,
}
}
/// The advisory lock file of the record root at `path`
/// (`<record-id>.lock` beside it).
fn record_lock_path(path: &Path) -> PathBuf {
path.with_file_name(format!(
"{}.lock",
path.file_name()
.map(|n| n.to_string_lossy().into_owned())
.unwrap_or_default()
))
}
/// Whether the lock file at `path` is held by a live build; an unreadable
/// probe errs on "held".
fn lock_held(path: &Path) -> bool {
crate::deb::session::lock_is_held(path).unwrap_or(true)
}
/// Determine the pkh cache directory (e.g. `~/.cache/pkh`), if project dirs
/// can be resolved on this platform.
pub fn cache_dir() -> Option<PathBuf> {
ProjectDirs::from("com", "pkh", "pkh").map(|d| d.cache_dir().to_path_buf())
}
/// Check whether a directory entry name is the apt keyring cache directory,
/// i.e. the legacy shared `pkh-keyrings` name or the per-uid
/// `pkh-keyrings-<uid>` used by current pkh versions.
fn is_keyring_dir_name(name: &str) -> bool {
if name == "pkh-keyrings" {
return true;
}
match name.strip_prefix("pkh-keyrings-") {
Some(uid) => !uid.is_empty() && uid.bytes().all(|b| b.is_ascii_digit()),
None => false,
}
}
/// Check whether a directory entry name is a residual pkh build directory,
/// i.e. it matches `pkh-<digits>` or `pkh-<digits>-<digits>`.
///
/// This deliberately rejects the `pkh-keyrings*` and `pkh-build-*` names so
/// they are not mistaken for residual build chroots.
fn is_residual_temp_dir(name: &str) -> bool {
let Some(rest) = name.strip_prefix("pkh-") else {
return false;
};
let mut parts = rest.splitn(2, '-');
let first = parts.next().unwrap_or("");
if first.is_empty() || !first.bytes().all(|b| b.is_ascii_digit()) {
return false;
}
match parts.next() {
None => true,
Some(second) => !second.is_empty() && second.bytes().all(|b| b.is_ascii_digit()),
}
}
/// Check whether a file name is a pkh build log, i.e. it ends in `.log` and
/// starts with one of the prefixes used by `pkh deb` logging: `deb-` (binary
/// builds), `build-` (source builds) or `pkh-` (the pre-identity placeholder
/// log that is renamed once the build identity is known).
fn is_log_file_name(name: &str) -> bool {
name.ends_with(".log")
&& (name.starts_with("deb-") || name.starts_with("build-") || name.starts_with("pkh-"))
}
/// Length of the `YYYYMMDDTHHMMSS` UTC timestamp embedded in log file names.
const LOG_TIMESTAMP_LEN: usize = 15;
/// Parse the UTC timestamp embedded in a build log file name.
///
/// Log names embed the timestamp as their final dash-separated component
/// (e.g. `deb-pkg-1.0-20260101T000000.log`, `build-src-1.0-20260101T000000.log`,
/// `pkh-20260101T000000.log`). Returns `None` when the name carries no
/// parseable timestamp.
fn embedded_log_timestamp(path: &Path) -> Option<SystemTime> {
let name = path.file_name()?.to_str()?;
let stem = name.strip_suffix(".log")?;
let candidate = stem.rsplit_once('-')?.1;
if candidate.len() != LOG_TIMESTAMP_LEN {
return None;
}
let ts = NaiveDateTime::parse_from_str(candidate, "%Y%m%dT%H%M%S").ok()?;
Some(ts.and_utc().into())
}
/// Recency of a build log, used to order logs for retention: later values are
/// more recent.
///
/// The timestamp embedded in the file name is authoritative. When the name
/// carries no parseable timestamp the file's modification time is used
/// instead, and finally the Unix epoch, so that undateable logs sort as the
/// oldest ones.
fn log_recency(path: &Path) -> SystemTime {
if let Some(ts) = embedded_log_timestamp(path) {
return ts;
}
fs::metadata(path)
.and_then(|m| m.modified())
.unwrap_or(SystemTime::UNIX_EPOCH)
}
/// Check whether a lockfile is safe to prune, i.e. it has not been modified
/// for at least [`LOCK_STALE_AFTER`].
///
/// A freshly touched lockfile may belong to a live concurrent `pkh` run, so
/// it must be left alone. If the modification time cannot be read the lock is
/// treated as ancient and pruned (best-effort cleanup of an unreadable
/// leftover); a modification time in the future (clock skew) counts as fresh.
fn is_stale_lock(path: &Path) -> bool {
let mtime = fs::metadata(path)
.and_then(|m| m.modified())
.unwrap_or(SystemTime::UNIX_EPOCH);
match mtime.elapsed() {
Ok(age) => age >= LOCK_STALE_AFTER,
Err(_) => false,
}
}
/// Unescape the octal escape sequences used in `/proc/mounts` mount points
/// (e.g. `\040` for space, `\011` for tab, `\012` for newline, `\134` for
/// backslash).
fn unescape_mountpath(s: &str) -> String {
let bytes = s.as_bytes();
let mut out = String::with_capacity(s.len());
let mut i = 0;
while i < bytes.len() {
if bytes[i] == b'\\' && i + 3 < bytes.len() {
let o1 = (bytes[i + 1] as char).to_digit(8);
let o2 = (bytes[i + 2] as char).to_digit(8);
let o3 = (bytes[i + 3] as char).to_digit(8);
if let (Some(a), Some(b), Some(c)) = (o1, o2, o3) {
let val = (a << 6) | (b << 3) | c;
out.push(val as u8 as char);
i += 4;
continue;
}
}
out.push(bytes[i] as char);
i += 1;
}
out
}
/// Parse `/proc/mounts`-style content and return mount points whose target is
/// located strictly *under* `path` (i.e. nested inside it, not `path` itself).
///
/// The list is returned ordered with the deepest paths first, so callers can
/// unmount children before their parents.
fn mounts_under_from_content(content: &str, path: &Path) -> Vec<String> {
let mut mounts: Vec<String> = Vec::new();
for line in content.lines() {
let mut fields = line.split_whitespace();
let _device = fields.next();
let Some(mountpoint) = fields.next() else {
continue;
};
let unescaped = unescape_mountpath(mountpoint);
let mp = PathBuf::from(&unescaped);
// starts_with is component-based, so /tmp/pkh-1234 is NOT considered
// under /tmp/pkh-123. We additionally exclude the dir itself.
if mp != path && mp.starts_with(path) {
mounts.push(unescaped);
}
}
// Deepest (longest) first so nested mounts are unmounted before parents.
mounts.sort_by_key(|m| std::cmp::Reverse(m.len()));
mounts
}
/// Discover all prune-able artifacts given the temp dir and cache dir roots.
fn discover_artifacts(temp_dir: &Path, cache_dir: Option<&Path>) -> Vec<Artifact> {
let mut artifacts = Vec::new();
if let Ok(entries) = fs::read_dir(temp_dir) {
for entry in entries.flatten() {
let path = entry.path();
let name = entry.file_name().to_string_lossy().into_owned();
if !path.is_dir() {
continue;
}
if is_residual_temp_dir(&name) {
artifacts.push(Artifact::TempDir(path));
} else if is_keyring_dir_name(&name) {
artifacts.push(Artifact::KeyringDir(path));
}
}
}
if let Some(cache) = cache_dir
&& let Ok(entries) = fs::read_dir(cache)
{
for entry in entries.flatten() {
let path = entry.path();
let name = entry.file_name().to_string_lossy().into_owned();
if name.ends_with(".lock") {
// Only stale lockfiles are prune-able; a fresh one may belong
// to a live concurrent pkh run (see [`is_stale_lock`]).
if is_stale_lock(&path) {
artifacts.push(Artifact::LockFile(path));
}
} else if name.ends_with("-buildd.tar.xz") {
artifacts.push(Artifact::Tarball(path));
}
}
}
// Build logs live in a dedicated subdirectory of the cache dir
if let Some(cache) = cache_dir {
let logs_dir = cache.join("logs");
if let Ok(entries) = fs::read_dir(&logs_dir) {
for entry in entries.flatten() {
let path = entry.path();
let name = entry.file_name().to_string_lossy().into_owned();
if path.is_file() && is_log_file_name(&name) {
artifacts.push(Artifact::LogFile(path));
}
}
}
}
artifacts
}
/// Compute the set of build log paths that should be removed for the given
/// options: all of them with `--all`, otherwise every log except the
/// [`KEEP_LOGS`] newest ones.
///
/// Recency is the timestamp embedded in each log's file name (falling back to
/// the file modification time when the name cannot be parsed; see
/// [`log_recency()`]). Log names sort by package and version first, so plain
/// lexicographic order is *not* chronological and must not be used here.
fn removable_logs(artifacts: &[Artifact], all: bool) -> std::collections::HashSet<&Path> {
let mut logs: Vec<&Path> = artifacts
.iter()
.filter_map(|a| match a {
Artifact::LogFile(p) => Some(p.as_path()),
_ => None,
})
.collect();
logs.sort_by_key(|p| log_recency(p));
if all {
return logs.into_iter().collect();
}
let keep = KEEP_LOGS.min(logs.len());
logs[..logs.len() - keep].iter().copied().collect()
}
/// Remove a path, first attempting a direct removal and only escalating to a
/// privileged `rm -rf` when the direct attempt fails.
///
/// User-writable artifacts (keyring dir, stale lockfiles) are removed directly
/// without shelling out. Residual build directories may contain root-owned
/// device nodes created via `mknod`, so the direct `remove_dir_all` will fail
/// with `EACCES` and we transparently retry with `sudo rm -rf` (or `rm -rf` when
/// already running as root).
fn remove_path(path: &Path) -> std::io::Result<()> {
let direct = if path.is_dir() {
fs::remove_dir_all(path)
} else {
fs::remove_file(path)
};
if direct.is_ok() {
return Ok(());
}
remove_with_privilege(path)
}
/// Run an `rm -rf <path>` removal, transparently using `sudo` when the current
/// process does not run as root. This is required because residual build
/// directories may contain root-owned device nodes.
fn remove_with_privilege(path: &Path) -> std::io::Result<()> {
let is_root = crate::utils::root::is_root().unwrap_or(false);
let mut cmd = if is_root {
Command::new("rm")
} else {
let mut c = Command::new("sudo");
c.arg("rm");
c
};
let status = cmd.arg("-rf").arg(path).status()?;
if status.success() {
Ok(())
} else {
Err(std::io::Error::other(format!(
"rm -rf {} failed with status {}",
path.display(),
status
)))
}
}
/// Unmount a single mount point, transparently using `sudo` when not root.
fn unmount(mountpoint: &str) -> std::io::Result<()> {
let is_root = crate::utils::root::is_root().unwrap_or(false);
let mut cmd = if is_root {
Command::new("umount")
} else {
let mut c = Command::new("sudo");
c.arg("umount");
c
};
let status = cmd.arg(mountpoint).status()?;
if status.success() {
Ok(())
} else {
Err(std::io::Error::other(format!(
"umount {} failed with status {}",
mountpoint, status
)))
}
}
/// Execute the prune operation against the given temp and cache roots,
/// removing residual pkh artifacts.
///
/// This is the testable core of [`prune()`]: callers pass in the temp
/// directory and optional cache directory to scan. Build sessions are
/// deliberately out of scope here — tests scan controlled roots, and a
/// concurrent test suite must never touch the live sessions under the
/// real sessions root. See the [module documentation](self) for what is
/// removed. Returns a [`PruneReport`] describing what was removed (or, in
/// dry-run mode, what would be removed).
pub fn prune_in(
temp_dir: &Path,
cache_dir: Option<&Path>,
options: PruneOptions,
) -> Result<PruneReport, Box<dyn std::error::Error>> {
Ok(apply(plan_in_roots(
temp_dir, cache_dir, None, None, None, options,
)))
}
/// Execute the prune operation against explicit roots.
///
/// `sessions_dir` opts in to build-record pruning, `tests_dir` to
/// test-run pruning and `apt_cache_dir` to shared apt-cache pruning (the
/// production [`prune()`] passes the real roots; tests pass controlled
/// directories or `None`).
pub fn prune_in_roots(
temp_dir: &Path,
cache_dir: Option<&Path>,
sessions_dir: Option<&Path>,
tests_dir: Option<&Path>,
apt_cache_dir: Option<&Path>,
options: PruneOptions,
) -> Result<PruneReport, Box<dyn std::error::Error>> {
Ok(apply(plan_in_roots(
temp_dir,
cache_dir,
sessions_dir,
tests_dir,
apt_cache_dir,
options,
)))
}
/// A plan produced by [`plan()`] or [`plan_in_roots()`]: the digest
/// report — groups with sizes, unless [`PruneOptions::force`] skipped
/// the sizing — and the artifacts [`apply()`] will remove.
pub struct PrunePlan {
report: PruneReport,
actionable: Vec<(Artifact, u64)>,
}
impl PrunePlan {
/// The digest: what would be removed, grouped by kind with sizes
/// (counts only with [`PruneOptions::force`]).
pub fn report(&self) -> &PruneReport {
&self.report
}
}
/// Discover and size the prunable artifacts against the given temp and
/// cache roots, removing nothing. This is the testable core of [`plan()`]:
/// callers pass in the temp directory and optional cache directory to
/// scan. Build sessions are deliberately out of scope here — tests scan
/// controlled roots, and a concurrent test suite must never touch the
/// live sessions under the real sessions root. Pair with [`apply()`].
pub fn plan_in_roots(
temp_dir: &Path,
cache_dir: Option<&Path>,
sessions_dir: Option<&Path>,
tests_dir: Option<&Path>,
apt_cache_dir: Option<&Path>,
options: PruneOptions,
) -> PrunePlan {
let mut artifacts = discover_artifacts(temp_dir, cache_dir);
if let Some(sessions_dir) = sessions_dir {
artifacts.extend(discover_sessions(sessions_dir, options.all));
}
if let Some(tests_dir) = tests_dir {
artifacts.extend(discover_test_runs(tests_dir, options.all));
}
if let Some(apt_cache_dir) = apt_cache_dir {
artifacts.extend(discover_apt_cache(apt_cache_dir, options.all));
}
let mut report = PruneReport {
dry_run: options.dry_run,
..Default::default()
};
let removable_logs = removable_logs(&artifacts, options.all);
let mut filtered: Vec<&Artifact> = Vec::new();
for artifact in &artifacts {
// Cached tarballs are only removed when --all is requested: they are
// expensive to re-download.
if matches!(artifact, Artifact::Tarball(_)) && !options.all {
continue;
}
// Build logs follow the retention policy computed above.
if let Artifact::LogFile(p) = artifact
&& !removable_logs.contains(p.as_path())
{
continue;
}
filtered.push(artifact);
}
// Sizes are measured during the plan (a removed tree has no size left
// to report) — unless --force skipped them: the plan then keeps the
// counts and loses the sizes, for a fast no-questions cleanup.
let sizes = if options.force {
vec![0; filtered.len()]
} else {
size_artifacts(&filtered)
};
let mut actionable: Vec<(Artifact, u64)> = Vec::new();
for (artifact, bytes) in filtered.iter().zip(sizes) {
report.removed.push(artifact.path().to_path_buf());
actionable.push(((*artifact).clone(), bytes));
// Group by kind in first-seen order.
if let Some(group) = report
.groups
.iter_mut()
.find(|group| group.kind == artifact.kind())
{
group.count += 1;
group.bytes += bytes;
} else {
report.groups.push(PruneGroup {
kind: artifact.kind(),
count: 1,
bytes,
});
}
}
report.total_bytes = report.groups.iter().map(|group| group.bytes).sum();
PrunePlan { report, actionable }
}
/// Progress observer of [`apply_with_progress`]: the removal of the
/// `idx`-th of `count` artifact (0-based) starts now, and `done` bytes
/// have been freed by the artifacts removed before it. Sizes are the
/// plan's estimates (measured before the removal; `--force` plans no
/// sizes and reports zeros).
pub type RemovalProgress<'a> = &'a dyn Fn(usize, usize, &Path, u64);
/// Execute a plan: remove the planned artifacts (unmounting the
/// filesystems under residual directories and session roots first) and
/// report what was removed and what failed. A dry-run plan removes
/// nothing and returns its digest unchanged.
pub fn apply(plan: PrunePlan) -> PruneReport {
apply_with_progress(plan, None)
}
/// [`apply`], reporting the removal as it progresses (the big session
/// trees take minutes): each artifact is announced just before it goes.
pub fn apply_with_progress(plan: PrunePlan, progress: Option<RemovalProgress<'_>>) -> PruneReport {
if plan.report.dry_run {
return plan.report;
}
let mut report = plan.report;
report.removed = Vec::new();
let count = plan.actionable.len();
let mut done = 0u64;
for (idx, (artifact, bytes)) in plan.actionable.iter().enumerate() {
let path = artifact.path();
if let Some(progress) = progress {
progress(idx, count, path, done);
}
// Residual build directories may hold bind-mounted /proc and overlay
// filesystems that must be unmounted before the directory can be
// removed, otherwise `rm -rf` will fail or leave dangling mounts.
// Session roots hold the same kind of mounts inside their chroot.
if let Artifact::TempDir(_) | Artifact::Session(_) = artifact {
let mounts_content = fs::read_to_string("/proc/mounts").unwrap_or_default();
for mp in mounts_under_from_content(&mounts_content, path) {
log::debug!("Unmounting {} under {}", mp, path.display());
if let Err(e) = unmount(&mp) {
log::warn!("Failed to unmount {}: {}", mp, e);
}
}
}
match remove_path(path) {
Ok(()) => {
report.removed.push(path.to_path_buf());
done += bytes;
}
Err(e) => report.failed.push((path.to_path_buf(), e.to_string())),
}
}
report
}
/// Plan the prune operation against the real roots: the digest without
/// removing anything. Pair with [`apply()`].
pub fn plan(options: PruneOptions) -> PrunePlan {
let temp_dir = std::env::temp_dir();
let cache = cache_dir();
plan_in_roots(
&temp_dir,
cache.as_deref(),
Some(&crate::deb::session::sessions_root()),
Some(&crate::test::runs_root()),
Some(&PathBuf::from(crate::deb::ephemeral::APT_CACHE_ROOT)),
options,
)
}
/// Execute the prune operation, removing residual pkh artifacts.
///
/// Convenience wrapper that uses the system temp directory and the pkh
/// cache directory. See the [module documentation](self) for what is
/// removed. Returns a [`PruneReport`] describing what was removed (or, in
/// dry-run mode, what would be removed).
pub fn prune(options: PruneOptions) -> Result<PruneReport, Box<dyn std::error::Error>> {
Ok(apply(plan(options)))
}
#[cfg(test)]
mod tests {
use super::*;
use std::fs;
use tempfile::tempdir;
/// Set a file's modification time (test helper).
fn set_mtime(path: &Path, mtime: SystemTime) {
let f = fs::OpenOptions::new().write(true).open(path).unwrap();
f.set_times(std::fs::FileTimes::new().set_modified(mtime))
.unwrap();
}
/// A modification time safely older than [`LOCK_STALE_AFTER`].
fn old_mtime() -> SystemTime {
SystemTime::now() - LOCK_STALE_AFTER - Duration::from_secs(3600)
}
#[test]
fn test_is_residual_temp_dir() {
assert!(is_residual_temp_dir("pkh-1700000000"));
assert!(is_residual_temp_dir("pkh-1700000000-1"));
assert!(is_residual_temp_dir("pkh-1700000000-12"));
// Keyrings and unshare work dirs must NOT match.
assert!(!is_residual_temp_dir("pkh-keyrings"));
assert!(!is_residual_temp_dir("pkh-keyrings-1000"));
assert!(!is_residual_temp_dir("pkh-build-1700000000"));
// Non-numeric / malformed names.
assert!(!is_residual_temp_dir("pkh-"));
assert!(!is_residual_temp_dir("pkh-abc"));
assert!(!is_residual_temp_dir("pkh-123-abc"));
assert!(!is_residual_temp_dir("pkh-123-"));
assert!(!is_residual_temp_dir("pkh"));
assert!(!is_residual_temp_dir("other-123"));
}
#[test]
fn test_is_keyring_dir_name() {
// Legacy shared name and current per-uid names are matched.
assert!(is_keyring_dir_name("pkh-keyrings"));
assert!(is_keyring_dir_name("pkh-keyrings-0"));
assert!(is_keyring_dir_name("pkh-keyrings-1000"));
// Everything else is not.
assert!(!is_keyring_dir_name("pkh-keyrings-"));
assert!(!is_keyring_dir_name("pkh-keyrings-abc"));
assert!(!is_keyring_dir_name("pkh-keyrings-1000-2"));
assert!(!is_keyring_dir_name("keyrings"));
assert!(!is_keyring_dir_name("pkh-1000"));
}
#[test]
fn test_is_log_file_name() {
// The three naming shapes produced by ui/deb.rs.
assert!(is_log_file_name("deb-hello-1.0-20260101T000000.log"));
assert!(is_log_file_name("build-src-1.0-20260101T000000.log"));
assert!(is_log_file_name("pkh-20260101T000000.log"));
// Everything else is not.
assert!(!is_log_file_name("deb-hello-1.0-20260101T000000.log.gz"));
assert!(!is_log_file_name("not-a-build.txt"));
assert!(!is_log_file_name("deb.txt"));
assert!(!is_log_file_name("logs.txt"));
assert!(!is_log_file_name("deb-"));
assert!(!is_log_file_name("other-thing.log"));
}
#[test]
fn test_log_recency_orders_by_embedded_timestamp() {
// The embedded timestamp decides, even when it disagrees with
// lexicographic order (zzz sorts after aaa).
let old = Path::new("/logs/deb-zzz-1.0-20200101T000000.log");
let new = Path::new("/logs/deb-aaa-2.0-20260101T000000.log");
assert!(log_recency(old) < log_recency(new));
// All three prefixes participate in the same timeline.
let build = Path::new("/logs/build-src-3.0-20230101T000000.log");
let placeholder = Path::new("/logs/pkh-20250101T000000.log");
assert!(log_recency(old) < log_recency(build));
assert!(log_recency(build) < log_recency(placeholder));
assert!(log_recency(placeholder) < log_recency(new));
}
#[test]
fn test_log_recency_falls_back_to_mtime() {
let temp = tempdir().unwrap();
// No parseable timestamp in the name, so the mtime decides.
let path = temp.path().join("deb-weird-name.log");
fs::write(&path, "log").unwrap();
let mtime = SystemTime::UNIX_EPOCH + Duration::from_secs(1_700_000_000);
set_mtime(&path, mtime);
assert_eq!(log_recency(&path), mtime);
// A missing file with an unparseable name sorts as the oldest.
assert_eq!(
log_recency(&PathBuf::from("/logs/deb-nope-nope.log")),
SystemTime::UNIX_EPOCH
);
}
#[test]
fn test_is_stale_lock() {
let temp = tempdir().unwrap();
let fresh = temp.path().join("fresh.tar.lock");
let stale = temp.path().join("stale.tar.lock");
fs::write(&fresh, "lock").unwrap();
fs::write(&stale, "lock").unwrap();
set_mtime(&stale, old_mtime());
assert!(!is_stale_lock(&fresh));
assert!(is_stale_lock(&stale));
}
#[test]
fn test_unescape_mountpath() {
assert_eq!(unescape_mountpath("/tmp/simple"), "/tmp/simple");
assert_eq!(unescape_mountpath("/tmp/with\\040space"), "/tmp/with space");
assert_eq!(unescape_mountpath("/tmp/with\\011tab"), "/tmp/with\ttab");
assert_eq!(
unescape_mountpath("/tmp/with\\134backslash"),
"/tmp/with\\backslash"
);
// Incomplete escape sequence is left untouched.
assert_eq!(unescape_mountpath("/tmp/with\\04"), "/tmp/with\\04");
}
#[test]
fn test_mounts_under_from_content() {
let path = PathBuf::from("/tmp/pkh-100");
let content = "\
proc /proc proc rw 0 0
proc /tmp/pkh-100/proc proc rw,bind 0 0
overlay /tmp/pkh-100/pkh-overlay/mnt overlay rw 0 0
overlay /tmp/pkh-1000/pkh-overlay/mnt overlay rw 0 0
none /tmp/other proc rw 0 0
";
let mounts = mounts_under_from_content(content, &path);
// Only mounts strictly under /tmp/pkh-100 should match, deepest first.
assert_eq!(mounts.len(), 2);
assert_eq!(mounts[0], "/tmp/pkh-100/pkh-overlay/mnt");
assert_eq!(mounts[1], "/tmp/pkh-100/proc");
}
#[test]
fn test_discover_artifacts() {
let temp = tempdir().unwrap();
let cache = tempdir().unwrap();
let temp_path = temp.path();
let cache_path = cache.path();
// Residual build dirs.
fs::create_dir_all(temp_path.join("pkh-1700000000")).unwrap();
fs::create_dir_all(temp_path.join("pkh-1700000001-2")).unwrap();
// Keyring dirs (legacy shared name and per-uid name).
fs::create_dir_all(temp_path.join("pkh-keyrings")).unwrap();
fs::create_dir_all(temp_path.join("pkh-keyrings-1000")).unwrap();
// Non-matching entries that should be ignored.
fs::create_dir_all(temp_path.join("pkh-build-1700000000")).unwrap();
fs::create_dir_all(temp_path.join("other-dir")).unwrap();
fs::write(temp_path.join("pkh-123-file"), "not a dir").unwrap();
// Cache entries.
fs::write(cache_path.join("noble-buildd.tar.xz"), "tarball").unwrap();
fs::write(cache_path.join("noble-amd64-buildd.tar.xz"), "tarball").unwrap();
// Only a stale lockfile is discovered; a fresh one could belong to a
// live concurrent run and must be ignored.
fs::write(cache_path.join("noble-buildd.tar.lock"), "lock").unwrap();
set_mtime(&cache_path.join("noble-buildd.tar.lock"), old_mtime());
fs::write(cache_path.join("noble-fresh.tar.lock"), "lock").unwrap();
fs::write(cache_path.join("stray.txt"), "ignore me").unwrap();
// Build logs: binary, source, and the pre-identity placeholder.
let logs_dir = cache_path.join("logs");
fs::create_dir_all(&logs_dir).unwrap();
fs::write(logs_dir.join("deb-hello-1.0-20260101T000000.log"), "log").unwrap();
fs::write(logs_dir.join("build-src-1.0-20260101T000000.log"), "log").unwrap();
fs::write(logs_dir.join("pkh-20260101T000000.log"), "log").unwrap();
fs::write(logs_dir.join("not-a-build.txt"), "ignore me").unwrap();
let artifacts = discover_artifacts(temp_path, Some(cache_path));
let mut temp_dirs = 0;
let mut keyring_dirs = 0;
let mut locks = 0;
let mut tarballs = 0;
let mut logs = 0;
for a in &artifacts {
match a {
Artifact::TempDir(_) => temp_dirs += 1,
Artifact::KeyringDir(_) => keyring_dirs += 1,
Artifact::LockFile(_) => locks += 1,
Artifact::Tarball(_) => tarballs += 1,
Artifact::LogFile(_) => logs += 1,
Artifact::Session(_) => {}
Artifact::TestRun(_) => {}
Artifact::AptCache(_) => {}
Artifact::SessionLock(_) => {}
}
}
assert_eq!(temp_dirs, 2);
assert_eq!(keyring_dirs, 2);
assert_eq!(locks, 1);
assert_eq!(tarballs, 2);
assert_eq!(logs, 3);
}
#[test]
fn test_prune_log_retention() {
let temp = tempdir().unwrap();
let cache = tempdir().unwrap();
let cache_path = cache.path();
let logs_dir = cache_path.join("logs");
fs::create_dir_all(&logs_dir).unwrap();
// Create KEEP_LOGS + 3 logs across all naming shapes; the 3 oldest
// (by embedded timestamp) should be pruned by default.
let total = KEEP_LOGS + 3;
let mut names = Vec::new();
for i in 0..total {
let ts = format!("20260101T{:06}", i);
let name = match i % 3 {
0 => format!("deb-pkg-1.0-{ts}.log"),
1 => format!("build-src-2.0-{ts}.log"),
_ => format!("pkh-{ts}.log"),
};
fs::write(logs_dir.join(&name), "log").unwrap();
names.push(name);
}
let report = prune_in(
temp.path(),
Some(cache_path),
PruneOptions {
dry_run: false,
all: false,
force: false,
},
)
.unwrap();
let mut removed_logs: Vec<String> = report
.removed
.iter()
.filter(|p| p.starts_with(&logs_dir))
.map(|p| p.file_name().unwrap().to_string_lossy().into_owned())
.collect();
removed_logs.sort();
// Exactly the three oldest logs (i = 0, 1, 2) were removed.
let mut expected: Vec<String> = names[..3].to_vec();
expected.sort();
assert_eq!(removed_logs, expected);
let remaining = fs::read_dir(&logs_dir).unwrap().count();
assert_eq!(remaining, KEEP_LOGS);
// --all removes every remaining log
let report = prune_in(
temp.path(),
Some(cache_path),
PruneOptions {
dry_run: false,
all: true,
force: false,
},
)
.unwrap();
let removed_logs = report
.removed
.iter()
.filter(|p| p.starts_with(&logs_dir))
.count();
assert_eq!(removed_logs, KEEP_LOGS);
assert_eq!(fs::read_dir(&logs_dir).unwrap().count(), 0);
}
#[test]
fn test_prune_log_retention_orders_by_timestamp_not_name() {
let temp = tempdir().unwrap();
let cache = tempdir().unwrap();
let cache_path = cache.path();
let logs_dir = cache_path.join("logs");
fs::create_dir_all(&logs_dir).unwrap();
// KEEP_LOGS new logs whose names sort lexicographically FIRST
// (package "aaa"), and older logs whose names sort LAST ("zzz" and a
// pkh- placeholder). Retention must follow the embedded timestamps:
// the aaa logs are kept and the lexicographically-later old logs are
// pruned, the opposite of what name ordering would do.
for i in 0..KEEP_LOGS {
fs::write(
logs_dir.join(format!("deb-aaa-1.0-20210101T{:06}.log", i)),
"log",
)
.unwrap();
}
let old_names = [
"deb-zzz-2.0-20200101T000000.log".to_string(),
"deb-zzz-2.0-20200101T000001.log".to_string(),
"build-zzz-3.0-20200101T000000.log".to_string(),
"pkh-20200101T000000.log".to_string(),
];
for name in &old_names {
fs::write(logs_dir.join(name), "log").unwrap();
}
let report = prune_in(
temp.path(),
Some(cache_path),
PruneOptions {
dry_run: false,
all: false,
force: false,
},
)
.unwrap();
let removed_logs: Vec<String> = report
.removed
.iter()
.filter(|p| p.starts_with(&logs_dir))
.map(|p| p.file_name().unwrap().to_string_lossy().into_owned())
.collect();
let mut removed_sorted = removed_logs.clone();
removed_sorted.sort();
let mut expected = old_names.clone();
expected.sort();
assert_eq!(removed_sorted, expected);
for i in 0..KEEP_LOGS {
assert!(
logs_dir
.join(format!("deb-aaa-1.0-20210101T{:06}.log", i))
.exists()
);
}
assert_eq!(fs::read_dir(&logs_dir).unwrap().count(), KEEP_LOGS);
}
#[test]
fn test_prune_dry_run_keeps_artifacts() {
let temp = tempdir().unwrap();
let cache = tempdir().unwrap();
let temp_path = temp.path();
let cache_path = cache.path();
let dir = temp_path.join("pkh-1700000000");
fs::create_dir_all(&dir).unwrap();
fs::create_dir_all(temp_path.join("pkh-keyrings")).unwrap();
fs::create_dir_all(temp_path.join("pkh-keyrings-1000")).unwrap();
fs::write(cache_path.join("noble-buildd.tar.xz"), "tarball").unwrap();
// A stale lockfile is reported; a fresh one may belong to a live
// concurrent run and is left out entirely.
fs::write(cache_path.join("noble-buildd.tar.lock"), "lock").unwrap();
set_mtime(&cache_path.join("noble-buildd.tar.lock"), old_mtime());
fs::write(cache_path.join("noble-live.tar.lock"), "lock").unwrap();
let report = prune_in(
temp_path,
Some(cache_path),
PruneOptions {
dry_run: true,
all: false,
force: false,
},
)
.unwrap();
// Dry run reports all discovered artifacts as "removed" without
// touching the filesystem.
assert!(report.dry_run);
assert!(report.removed.iter().any(|p| p == &dir));
assert!(
report
.removed
.iter()
.any(|p| p == &temp_path.join("pkh-keyrings"))
);
assert!(
report
.removed
.iter()
.any(|p| p == &temp_path.join("pkh-keyrings-1000"))
);
assert!(
report
.removed
.iter()
.any(|p| p == &cache_path.join("noble-buildd.tar.lock"))
);
// The fresh lockfile is not reported and survives the dry run.
assert!(
!report
.removed
.iter()
.any(|p| p == &cache_path.join("noble-live.tar.lock"))
);
// Tarballs require --all, so not reported here.
assert!(
!report
.removed
.iter()
.any(|p| p == &cache_path.join("noble-buildd.tar.xz"))
);
// Nothing was actually removed.
assert!(dir.exists());
assert!(temp_path.join("pkh-keyrings").exists());
assert!(temp_path.join("pkh-keyrings-1000").exists());
assert!(cache_path.join("noble-buildd.tar.xz").exists());
assert!(cache_path.join("noble-buildd.tar.lock").exists());
assert!(cache_path.join("noble-live.tar.lock").exists());
}
#[test]
fn test_prune_removes_user_writable_cache() {
// Removing the keyring dir and stale lockfiles does not require
// privilege escalation since they are user-writable; exercise the real
// prune_in() against controlled temp roots without needing sudo.
let temp = tempdir().unwrap();
let cache = tempdir().unwrap();
let temp_path = temp.path();
let cache_path = cache.path();
fs::create_dir_all(temp_path.join("pkh-keyrings")).unwrap();
fs::create_dir_all(temp_path.join("pkh-keyrings-1000")).unwrap();
fs::write(cache_path.join("noble-buildd.tar.lock"), "lock").unwrap();
set_mtime(&cache_path.join("noble-buildd.tar.lock"), old_mtime());
fs::write(cache_path.join("noble-fresh.tar.lock"), "lock").unwrap();
fs::write(cache_path.join("noble-buildd.tar.xz"), "tarball").unwrap();
let report = prune_in(
temp_path,
Some(cache_path),
PruneOptions {
dry_run: false,
all: false,
force: false,
},
)
.unwrap();
assert!(!report.dry_run);
assert!(
report
.removed
.iter()
.any(|p| p == &temp_path.join("pkh-keyrings"))
);
assert!(
report
.removed
.iter()
.any(|p| p == &temp_path.join("pkh-keyrings-1000"))
);
assert!(
report
.removed
.iter()
.any(|p| p == &cache_path.join("noble-buildd.tar.lock"))
);
assert!(!temp_path.join("pkh-keyrings").exists());
assert!(!temp_path.join("pkh-keyrings-1000").exists());
assert!(!cache_path.join("noble-buildd.tar.lock").exists());
// The fresh lockfile may belong to a live run and is preserved.
assert!(cache_path.join("noble-fresh.tar.lock").exists());
// Tarball is preserved when --all is not set.
assert!(cache_path.join("noble-buildd.tar.xz").exists());
}
#[test]
fn test_prune_all_removes_tarballs() {
let temp = tempdir().unwrap();
let cache = tempdir().unwrap();
let cache_path = cache.path();
fs::write(cache_path.join("noble-buildd.tar.xz"), "tarball").unwrap();
fs::write(cache_path.join("noble-amd64-buildd.tar.xz"), "tarball").unwrap();
fs::write(cache_path.join("noble-buildd.tar.lock"), "lock").unwrap();
set_mtime(&cache_path.join("noble-buildd.tar.lock"), old_mtime());
let report = prune_in(
temp.path(),
Some(cache_path),
PruneOptions {
dry_run: false,
all: true,
force: false,
},
)
.unwrap();
assert!(
report
.removed
.iter()
.any(|p| p == &cache_path.join("noble-buildd.tar.xz"))
);
assert!(
report
.removed
.iter()
.any(|p| p == &cache_path.join("noble-amd64-buildd.tar.xz"))
);
assert!(
report
.removed
.iter()
.any(|p| p == &cache_path.join("noble-buildd.tar.lock"))
);
assert!(!cache_path.join("noble-buildd.tar.xz").exists());
assert!(!cache_path.join("noble-amd64-buildd.tar.xz").exists());
assert!(!cache_path.join("noble-buildd.tar.lock").exists());
}
#[test]
fn test_prune_keeps_fresh_lockfile_removes_stale_one() {
// The chroot tarball download uses the lockfile's existence as a
// mutual-exclusion signal, so a lockfile fresh enough to belong to a
// live concurrent pkh run must survive pruning.
let temp = tempdir().unwrap();
let cache = tempdir().unwrap();
let cache_path = cache.path();
let fresh = cache_path.join("noble-live.tar.lock");
let stale = cache_path.join("noble-abandoned.tar.lock");
fs::write(&fresh, "lock").unwrap();
fs::write(&stale, "lock").unwrap();
set_mtime(&stale, old_mtime());
let report = prune_in(
temp.path(),
Some(cache_path),
PruneOptions {
dry_run: false,
all: false,
force: false,
},
)
.unwrap();
assert!(report.removed.iter().any(|p| p == &stale));
assert!(!report.removed.iter().any(|p| p == &fresh));
assert!(fresh.exists());
assert!(!stale.exists());
// Dry-run reports the same: only the stale lockfile.
fs::write(&stale, "lock").unwrap();
set_mtime(&stale, old_mtime());
let report = prune_in(
temp.path(),
Some(cache_path),
PruneOptions {
dry_run: true,
all: false,
force: false,
},
)
.unwrap();
assert!(report.removed.iter().any(|p| p == &stale));
assert!(!report.removed.iter().any(|p| p == &fresh));
assert!(fresh.exists());
assert!(stale.exists());
}
#[test]
fn test_prune_removes_user_writable_temp_dirs() {
// Residual build dirs created by the tests are user-writable, so the
// direct fs::remove_dir_all path should succeed without sudo.
let temp = tempdir().unwrap();
let temp_path = temp.path();
fs::create_dir_all(temp_path.join("pkh-1700000000")).unwrap();
fs::create_dir_all(temp_path.join("pkh-1700000001-2")).unwrap();
let report = prune_in(temp_path, None, PruneOptions::default()).unwrap();
assert!(
report
.removed
.iter()
.any(|p| p == &temp_path.join("pkh-1700000000"))
);
assert!(
report
.removed
.iter()
.any(|p| p == &temp_path.join("pkh-1700000001-2"))
);
assert!(!temp_path.join("pkh-1700000000").exists());
assert!(!temp_path.join("pkh-1700000001-2").exists());
}
/// The removal progress announces every artifact just before it goes,
/// with the cumulative bytes of the artifacts already removed: the
/// terminal drives its bar from it while the big trees take minutes.
#[test]
fn apply_reports_each_removal_with_cumulative_sizes() {
let temp = tempdir().unwrap();
let temp_path = temp.path();
let big = temp_path.join("pkh-1700000000");
fs::create_dir_all(&big).unwrap();
fs::write(big.join("obj.o"), vec![0u8; 3000]).unwrap();
let small = temp_path.join("pkh-1700000001");
fs::create_dir_all(&small).unwrap();
fs::write(small.join("o"), vec![0u8; 5]).unwrap();
let plan = plan_in_roots(temp_path, None, None, None, None, PruneOptions::default());
assert_eq!(plan.report().total_bytes, 3005);
let events = std::sync::Mutex::new(Vec::new());
let report = apply_with_progress(
plan,
Some(&|idx, count, path, done| {
events
.lock()
.unwrap()
.push((idx, count, path.to_path_buf(), done));
}),
);
assert_eq!(report.removed.len(), 2);
assert!(!big.exists());
assert!(!small.exists());
let events = events.into_inner().unwrap();
assert_eq!(events.len(), 2);
// The first announcement opens the pass; the second names the
// other artifact and carries the first one's size (discovery
// order is the temp dir's read order, not ours).
assert_eq!(events[0].0, 0);
assert_eq!(events[0].1, 2);
assert_eq!(events[0].3, 0);
assert_eq!(events[1].0, 1);
assert_eq!(events[1].1, 2);
assert_ne!(events[1].2, events[0].2);
let expected_done = if events[0].2 == big { 3000 } else { 5 };
assert_eq!(events[1].3, expected_done);
}
#[test]
fn test_prune_empty_report_when_nothing_found() {
let temp = tempdir().unwrap();
let cache = tempdir().unwrap();
// No pkh artifacts at all.
let report = prune_in(temp.path(), Some(cache.path()), PruneOptions::default()).unwrap();
assert!(report.is_empty());
}
/// Write a minimal record manifest with the given `last_used` under
/// `sessions/<slug>/<record>/`.
fn write_session_manifest(session_dir: &Path, last_used: &str) {
fs::create_dir_all(session_dir).unwrap();
fs::write(
session_dir.join("manifest.json"),
format!(r#"{{ "last_used": "{last_used}" }}"#),
)
.unwrap();
}
/// Record pruning: stale and corrupt records go by default, live ones
/// stay; `--all` takes everything.
#[test]
fn test_session_pruning_retention() {
let temp = tempdir().unwrap();
let sessions = temp.path().join("sessions");
let slug = sessions.join("pkh-noble-amd64-1c0ffee0");
let stale = slug.join("20200101T000000");
let fresh = slug.join("20260929T000000");
let corrupt = slug.join("20260929T000001");
write_session_manifest(&stale, "2020-01-01T00:00:00+00:00");
write_session_manifest(
&fresh,
&(chrono::Utc::now() - chrono::Duration::hours(1)).to_rfc3339(),
);
fs::create_dir_all(&corrupt).unwrap();
fs::write(corrupt.join("manifest.json"), "not json").unwrap();
let report = prune_in_roots(
temp.path(),
None,
Some(&sessions),
None,
None,
PruneOptions::default(),
)
.unwrap();
assert!(report.removed.contains(&stale));
assert!(report.removed.contains(&corrupt));
assert!(fresh.exists(), "a live record is not pruned by default");
assert!(!stale.exists());
assert!(!corrupt.exists());
// --all removes everything, including the live record.
let report = prune_in_roots(
temp.path(),
None,
Some(&sessions),
None,
None,
PruneOptions {
all: true,
..Default::default()
},
)
.unwrap();
assert!(report.removed.contains(&fresh));
assert!(!fresh.exists());
}
/// `--all` removes fresh records and staged trees (that is where the
/// space is), but a record whose lock a live build holds is never
/// pruned, and neither is its lock file.
#[test]
fn test_session_pruning_all_spares_records_a_live_build_holds() {
use std::os::unix::io::AsRawFd;
let temp = tempdir().unwrap();
let sessions = temp.path().join("sessions");
let slug = sessions.join("pkh-noble-amd64-1c0ffee0");
let free = slug.join("20260929T000000");
let busy = slug.join("20260929T000001");
write_session_manifest(&free, &chrono::Utc::now().to_rfc3339());
write_session_manifest(&busy, &chrono::Utc::now().to_rfc3339());
// Hold the busy record's lock for the whole test: a live build
// owns it.
let lock_path = slug.join("20260929T000001.lock");
let held = fs::OpenOptions::new()
.create(true)
.truncate(true)
.write(true)
.open(&lock_path)
.unwrap();
unsafe {
libc::flock(held.as_raw_fd(), libc::LOCK_EX | libc::LOCK_NB);
}
let report = prune_in_roots(
temp.path(),
None,
Some(&sessions),
None,
None,
PruneOptions::default(),
)
.unwrap();
assert!(report.removed.is_empty(), "fresh records stay by default");
// --all takes the unheld record and leaves the held one (and its
// lock) alone.
let report = prune_in_roots(
temp.path(),
None,
Some(&sessions),
None,
None,
PruneOptions {
all: true,
..Default::default()
},
)
.unwrap();
assert!(report.removed.contains(&free));
assert!(!free.exists());
assert!(busy.exists(), "a held record survives --all");
assert!(lock_path.exists(), "a held lock survives --all");
}
/// The report groups the removed paths by kind with their sizes, so
/// the output is a digest instead of a wall of paths.
#[test]
fn test_prune_report_groups_by_kind_with_sizes() {
let temp = tempdir().unwrap();
let cache = tempdir().unwrap();
let dir = temp.path().join("pkh-1700000000");
fs::create_dir_all(&dir).unwrap();
fs::write(dir.join("obj.o"), vec![0u8; 3000]).unwrap();
let lockfile = cache.path().join("noble-buildd.tar.lock");
fs::write(&lockfile, "lock").unwrap();
set_mtime(&lockfile, old_mtime());
let report = prune_in(
temp.path(),
Some(cache.path()),
PruneOptions {
dry_run: true,
..Default::default()
},
)
.unwrap();
let dirs = report
.groups
.iter()
.find(|g| g.kind == "residual build directory")
.expect("residual directories are grouped");
assert_eq!(dirs.count, 1);
assert_eq!(dirs.bytes, 3000);
// Count 1: the label stays singular; pluralization is covered by
// the labels test below.
assert_eq!(dirs.label(), "residual build directory");
let locks = report
.groups
.iter()
.find(|g| g.kind == "stale lockfile")
.expect("stale lockfiles are grouped");
assert_eq!(locks.count, 1);
assert_eq!(locks.bytes, 4);
assert_eq!(locks.label(), "stale lockfile");
assert_eq!(report.total_bytes, 3004);
assert_eq!(format_bytes(3004), "2.9 KiB");
}
#[test]
fn test_format_bytes_and_group_labels() {
assert_eq!(format_bytes(0), "0 B");
assert_eq!(format_bytes(512), "512 B");
assert_eq!(format_bytes(2048), "2.0 KiB");
assert_eq!(format_bytes(5 * 1024 * 1024), "5.0 MiB");
assert_eq!(format_bytes(3 * 1024 * 1024 * 1024), "3.0 GiB");
// /proc/kcore inside a bind-mounted chroot /proc reports exactly
// this size; the mount-crossing walk keeps it out of the totals.
assert_eq!(format_bytes(140_737_477_959_680), "128.0 TiB");
let group = PruneGroup {
kind: "shared apt download cache",
count: 3,
bytes: 10,
};
assert_eq!(group.label(), "shared apt download caches");
let group = PruneGroup {
kind: "residual build directory",
count: 18,
bytes: 10,
};
assert_eq!(group.label(), "residual build directories");
}
/// `pkh test` run directories follow the retention window: stale
/// runs go, fresh runs stay, `--all` takes everything.
#[test]
fn prunes_test_runs_by_retention() {
let temp = tempdir().unwrap();
let tests = temp.path().join("tests");
let stale = tests.join("pkg-noble-amd64-20260101T000000");
let fresh = tests.join("pkg-noble-amd64-20991231T235959");
fs::create_dir_all(&stale).unwrap();
fs::create_dir_all(&fresh).unwrap();
// Directories open read-only (write-open is a file-only mode):
// futimens still applies through the read handle.
let dir = fs::File::open(&stale).unwrap();
dir.set_times(
std::fs::FileTimes::new()
.set_modified(SystemTime::now() - std::time::Duration::from_secs(8 * 24 * 60 * 60)),
)
.unwrap();
let report = prune_in_roots(
temp.path(),
None,
None,
Some(&tests),
None,
PruneOptions::default(),
)
.unwrap();
assert!(report.removed.contains(&stale));
assert!(!stale.exists());
assert!(fresh.exists(), "a fresh test run is not pruned");
// --all removes every run.
let report = prune_in_roots(
temp.path(),
None,
None,
Some(&tests),
None,
PruneOptions {
all: true,
..Default::default()
},
)
.unwrap();
assert!(report.removed.contains(&fresh));
assert!(!fresh.exists());
}
/// The shared apt download caches follow the retention window: a
/// cache untouched for over a week goes, one in recent use stays,
/// `--all` takes everything.
#[test]
fn prunes_stale_apt_caches_by_retention() {
let temp = tempdir().unwrap();
let apt_cache = temp.path().join("apt-cache");
let stale = apt_cache.join("noble-amd64");
let fresh = apt_cache.join("sid-riscv64");
for dir in [&stale, &fresh] {
fs::create_dir_all(dir.join("partial")).unwrap();
fs::write(dir.join("some-deb_1.0_amd64.deb"), "deb").unwrap();
}
// Directories open read-only (write-open is a file-only mode):
// futimens still applies through the read handle.
let dir = fs::File::open(&stale).unwrap();
dir.set_times(
std::fs::FileTimes::new()
.set_modified(SystemTime::now() - std::time::Duration::from_secs(8 * 24 * 60 * 60)),
)
.unwrap();
let report = prune_in_roots(
temp.path(),
None,
None,
None,
Some(&apt_cache),
PruneOptions::default(),
)
.unwrap();
assert!(report.removed.contains(&stale));
assert!(!stale.exists());
assert!(fresh.exists(), "a recently used cache is not pruned");
// --all removes every cache.
let report = prune_in_roots(
temp.path(),
None,
None,
None,
Some(&apt_cache),
PruneOptions {
all: true,
..Default::default()
},
)
.unwrap();
assert!(report.removed.contains(&fresh));
assert!(!fresh.exists());
}
/// Orphaned record lock files are pruned (unless held by a live
/// build); locks whose record root still exists are never touched.
/// The pre-record layout's locks at the sessions root follow the same
/// rules.
#[test]
fn test_session_lock_pruning() {
use std::os::unix::io::AsRawFd;
let temp = tempdir().unwrap();
let sessions = temp.path().join("sessions");
let slug = sessions.join("pkh-noble-amd64-1c0ffee0");
let orphan_free = slug.join("20260929T000000.lock");
let orphan_held = slug.join("20260929T000001.lock");
let rooted = slug.join("20260929T000002.lock");
let legacy = sessions.join("legacy-slug.lock");
fs::create_dir_all(&slug).unwrap();
fs::write(&orphan_free, b"").unwrap();
fs::write(&orphan_held, b"").unwrap();
fs::write(&rooted, b"").unwrap();
fs::create_dir_all(slug.join("20260929T000002")).unwrap();
fs::write(&legacy, b"").unwrap();
// Hold one orphaned lock, as a live build would.
let holder = fs::OpenOptions::new()
.read(true)
.open(&orphan_held)
.unwrap();
unsafe {
libc::flock(holder.as_raw_fd(), libc::LOCK_EX | libc::LOCK_NB);
}
let report = prune_in_roots(
temp.path(),
None,
Some(&sessions),
None,
None,
PruneOptions::default(),
)
.unwrap();
assert!(report.removed.contains(&orphan_free));
assert!(!orphan_free.exists());
assert!(report.removed.contains(&legacy));
assert!(!legacy.exists());
assert!(orphan_held.exists(), "a held lock belongs to a live build");
assert!(rooted.exists(), "a lock guarding an existing record stays");
// --all takes the held orphan too (the record it guarded is gone).
drop(holder);
let report = prune_in_roots(
temp.path(),
None,
Some(&sessions),
None,
None,
PruneOptions {
all: true,
..Default::default()
},
)
.unwrap();
assert!(report.removed.contains(&orphan_held));
assert!(!orphan_held.exists());
}
}