Files

T

vhaudiquet 87f5b31602 test: add the e2e package harness

Exercises pkh the way it is used in anger: 63 real source packages
(from hello and micro to dracut, grub2 and the kernel) run through
pull -> chlog --rebuild -> build -> deb, each stage timed, logged and
checked for its artifacts, with the catalog alternating between debian
and ubuntu within every category.

run.py keeps runs self-contained (per-package logs, progressive
results.json, a markdown summary), pulls in parallel while builds run
sequentially to keep pkh's shared chroot/apt/ccache caches warm,
resumes interrupted runs reusing the stages that already succeeded,
and marks known-broken entries as xfail with a reason.

The first catalog run caught four pkh defects within an hour - the
unstable/sid suite alias, the bz2 origs, the hard link extraction and
the wrapper unwrapping - which is exactly what it is for.

2026-10-01 22:24:07 +02:00

4.6 KiB

Raw Blame History

pkh end-to-end package harness

Exercises pkh the way it is used in anger: real packages, the whole workflow. For every package in catalog.yml it runs

pkh pull <name> --archive     # sources from the Debian/Ubuntu archive
pkh chlog --rebuild           # no-change rebuild entry (1.0-1 -> 1.0-1build1)
pkh build                     # .dsc + _source.changes next to the tree
pkh deb                       # binary packages in an ephemeral chroot

The catalog spans ~60 source packages: smoke-test smalls (hello, micro, jq...), core utilities, server software, databases, desktop stacks, toolchains and the weird corners — dracut, opensbi, grub2, a cross u-boot, the kbuild-based kbuild, and (opt-in, hours each) gcc-14, glibc, rustc, libreoffice, firefox, the Ubuntu linux kernel...

Prerequisites

A pkh binary: the harness prefers the repo build (target/release/pkh, then target/debug/pkh), falls back to pkh on PATH; override with --pkh PATH or build first with --build-pkh.
The pkh runtime toolchain: mmdebstrap, sudo (passwordless, for the unshare chroot builds), dpkg-dev, distro-info data, and network access to the Debian and Ubuntu archives.
Python 3 only (stdlib; PyYAML is used when installed, with a built-in fallback parser for the catalog's YAML subset).
Disk: tens of GB under runs/ for trees and artifacts, plus pkh's own caches (/var/tmp/pkh); expect a full default-tier run to take many hours — the tiers exist so you never have to.

Usage

./run.py --list                    # what would run, with tiers and notes
./run.py --dry-run                 # the exact commands, nothing executed
./run.py                           # smoke + standard tiers
./run.py --tier long               # the multi-hour builds
./run.py --tier all                # everything
./run.py --only hello,dracut       # exact names
./run.py --category database       # or server, desktop, boot-firmware...
./run.py --tag cross --arch arm64  # tag/arch sweeps
./run.py --stages pull             # catalog sanity: can we pull everything?
./run.py --resume runs/<id>/results.json   # continue an interrupted run

Useful global pass-throughs: --dist, --series, --arch, --cross, --pocket, --ppa user/ppa (deb dependencies), --jobs N, --no-ccache, --git (pull through the VCS instead of the archive).

Exit codes: 0 nothing unexpected failed, 1 at least one package failed (or xpassed under --strict), 2 usage/catalog error, 130 interrupted.

Selection: tiers, categories, tags

Entries carry a tier: smoke (minutes), standard (the default, up to ~1-2h each), long (hours: compilers, browsers, the kernel — only run when asked). Default selection is smoke + standard. On top of that, filters compose: --category, --tag, --only, --skip.

An entry can be marked xfail: <reason> when it is known-broken; it then reports xfail on failure and xpass on unexpected success (xpass fails the run only under --strict).

What a run produces

runs/<id>/
  logs/<pkg>.log      # full output of every stage of that package
  <pkg>/<pkg>/        # the pulled source tree
  <pkg>/*.dsc ...     # build artifacts, next to the tree as pkh puts them
  results.json        # machine-readable, written progressively
  results.md          # the summary table

The run directory is per invocation; --resume <id>/results.json reuses it, keeps the recorded outcomes and only reruns packages that did not finish. Already-pulled trees are kept (pkh pull refuses existing directories anyway — delete runs/<id>/<pkg>/ to force a re-pull), and stages that already succeeded are reused rather than re-run, so a resumed chlog does not bump the version twice; change --dist/--series/--arch/--cross and everything re-runs.

Pulls run a few at a time (--pull-jobs), the build pipeline is sequential by design: one heavy build at a time keeps pkh's shared chroot tarball, apt and ccache caches warm, so repeat runs get dramatically faster.

Each stage gets the package's timeout budget (tier default, overridable per entry with timeout:); on timeout the whole build process group is killed and the package is marked timeout.

Adding packages

Edit catalog.yml. Fields are documented at the top of the file; keep source names as they exist in the archive — not binary names (make-dfsg and rust-ripgrep, not make and ripgrep; xz-utils, not xz; postgresql-17, not postgresql). The file stays within a small YAML subset so the stdlib fallback parser can read it: flat mappings, inline lists, no colons in values.

4.6 KiB Raw Blame History