pkh/README.md

pkh

pkh is a packaging helper for Debian/Ubuntu packages.

Installation

From crates.io:

cargo install pkh

Or build from source (the same system packages are needed either way):

sudo apt install pkg-config libssl-dev libgpg-error-dev libgpgme-dev
git clone https://git.vhaudiquet.fr/vhaudiquet/pkh.git
cd pkh
cargo install --path .

At runtime pkh shells out to the Debian packaging toolchain (git, dpkg-dev, quilt, mmdebstrap, lintian, pristine-tar, autopkgtest, ...): install the ones your workflows use, or build the classic snap from snap/snapcraft.yaml (snapcraft pack), which carries them.

Usage and features

Basic concepts

pkh aims at wrapping the different debian tools and workflows (git, git-ubuntu, pull-debian-sources, pull-lp-sources, pull-ppa-sources, dch, dpkg-buildpackage, sbuild, dpkg-source, quilt, ...) into one tool, that would have the same interface for everything, while being smarter at integrating all workflows.

Thus, pkh uses similar options for all subcommands (with very few command-specific options):

Options:
  -s, --series <series>    Target package distribution series
  -d, --dist <dist>        Target package distribution (debian, ubuntu)
  -v, --version <version>  Target package version
  -a, --arch <arch>        Target architecture (amd64, arm64, riscv64, ...)
  -p, --pocket <pocket>    Target distribution pocket (updates, security, proposed, ...)
      --ppa <ppa>          Do the action in/for a specific PPA

Commands and workflows include:

Commands:
  new    Scaffold a new Debian source package (buildable right away)
  pull   Pull a source package from the archive or git
  chlog  Auto-generate changelog entry, editing it, committing it afterwards
  build  Build the source package (into a .dsc)
  put    Upload the built source package to a PPA
  deb    Build the source package into binary package (.deb)
  test   Run the package's DEP-8 as-installed tests (autopkgtests)
  lint   Lint the package (lintian wrapper + pkh-native checks)
  prune  Prune residual pkh build artifacts and caches
  help   Print this message or the help of the given subcommand(s)

Options:
  -h, --help     Print help
  -V, --version  Print version

Examples

A typical workflow to patch an Ubuntu package hello on the development release could be:

# Obtain the package source
git ubuntu clone hello
cd hello
# Apply the patch to the package
...
dpkg-source --commit
# Increment version number
dch
# Test that the patch builds
git ubuntu export-orig
dpkg-buildpackage -S -I -i -nc -d
sbuild --dist resolute --arch amd64 ../hello_xxx.dsc
# Upload the package to a ppa
dput ppa:user/hello_xxx ../hello_xxx_source.changes
# Commit the changes to git
git add debian/patches/xxx.patch
git commit -m "Applied patch xxx"
git add debian/changelog
git commit -m "changelog"
git checkout -b xxx
git push xxx user-fork

That is a lot of different tools and operations. With pkh, the same workflow:

# Obtain the package source (and orig tarball)
pkh pull hello # needs -d ubuntu if you are not running Ubuntu
# Apply the patch to the package
...
git add debian/patches/xxx.patch
git commit -m "Applied patch xxx"
pkh chlog
git add debian/changelog
git commit -m "d/changelog"
# Test that the package builds
pkh build
pkh deb
# Upload the package to a ppa
pkh put --ppa user/hello_xxx
# Push the commits to your fork
git push xxx user-fork

pkh chlog edits debian/changelog by default; point it at another file with --file <path> (-f).

pkh chlog walks the commit history back to the previous changelog entry. Without a version tag, that search is capped at the last 500 commits (--limit <n>, 0 = unlimited): beyond the cap, the entry simply starts at the oldest commit searched, which keeps the run fast on large histories.

Incremental builds (build records)

Every pkh deb build runs in its own immutable record under /var/tmp/pkh/sessions/<target>/<id>/: the bootstrapped chroot, the installed build dependencies and the build artifacts of the staged tree. --incremental forks a previous record — a full copy, the base is never modified — so a failed run can never destroy the last-good environment it was built from:

pkh deb                 # fully ephemeral: nothing is kept, even on
                        # failure or interrupt
pkh deb --incremental   # fork the newest successful record of the
                        # target (last five usable): the chroot, build
                        # deps and, at the same version, the objects
                        # are reused; only what changed is recompiled.
                        # The result is always kept (success or
                        # failure) for the next fork
pkh deb --incremental <id>
                        # fork that exact record (see `pkh deb list`);
                        # a failed record is fine if its environment is
pkh deb list            # the records of this tree: id, outcome,
                        # version, fork lineage, age
pkh prune               # garbage-collect records (7-day retention)

Each target (series/arch) keeps its last five records — older ones are pruned as new ones are created, and pkh prune removes those untouched for a week. pkh deb --incremental never forks a record built for a different series/architecture, and a plain pkh deb still never reuses anything: it builds in a throwaway record that is discarded on success, failure and interrupt alike.

Packages whose debian/rules keeps an input-untracked stamp cache (the kernels' debian/stamps) declare it in the quirks data (resume_clear): forked builds drop the cache so the wrapper steps — including the kernel's flavour config export — re-run over the incremental inner builds, and a config edit is never silently ignored. If a build died hard (OOM kill), the next fork unmounts the leftover state first (escalating through sudo when needed).

Running the DEP-8 tests

pkh test runs the package's as-installed tests (debian/tests/control, declared by Testsuite: autopkgtest in debian/control) through a native runner on a pkh testbed, and reports one verdict per test:

pkh test                 # test this tree: builds it first if needed
pkh test --mode local    # on the current context, no isolation
pkh test --mode vm --image img.qcow2   # in a VM, via autopkgtest
pkh test --json --fail-on skip         # CI-shaped
pkh test --list-tests    # what does this package test?

By default the tests run in an ephemeral unshare chroot bootstrapped from the same cached tarballs as pkh deb, against the binaries pkh deb just built next to the tree (stale or missing output is rebuilt first; --no-build refuses, --debs overrides). Pockets and PPAs resolve test dependencies like in pkh deb (-p, --ppa), and --test-name, --skip-test, --shell-fail and --setup-commands cover the day-to-day debugging loop. Tests needing a real machine (isolation-machine, needs-reboot) run only in --mode vm, which execs the installed autopkgtest with its qemu runner.

Exit codes: 0 all tests passed (skips and flaky allowed), 1 at least one test failed (or a --fail-on trigger), 2 runtime errors (no declared tests, no usable testbed, ...).

Future improvement ideas

• pull: try to fetch the correct git branch for series on Debian
• deb: asynchronous build, detachable and monitorable
• put: allow uploads to Debian or Ubuntu archives
• pull: cache Sources.gz files to improve speed
• pull: 'pkh pull' in a package tree should git pull and re-fetch orig tgz