vhaudiquet
d927dc93ab
Two failures from resumed kernel sessions: A resumed build re-runs debian/rules build, but the kernels keep their wrapper-step stamps in debian/stamps and the flavour config rule there exports .config from the annotations with no config prerequisite: the resumed build silently kept the previous attempt's configuration and a config edit never reached the .deb. Dropping the stamp cache unconditionally would impose the kernel packaging's shape on every package, so it is data-driven instead: packages declare the caches under a resume_clear quirk, and resumed builds remove those tree-relative paths before the build. The inner kbuild keeps its own incremental state, so only the cheap wrapper passes re-run and config-affected objects recompile. A SIGKILLed build (OOM) leaves its overlay mounts and /proc bind mount behind, and overlayfs creates root-owned work state inside the workdir: the next plain build (recording is always on, it replaces the session of its identity) failed to clear the leftovers with a permission error. Session clearing now unmounts everything under each entry at depth (re-reading /proc/mounts, tolerating mount stacks from consecutive crashes) and escalates through sudo; when the tree still cannot be cleared, the build reports it and continues without a session instead of layering over a half-cleared one.
180 lines
6.5 KiB
Markdown
180 lines
6.5 KiB
Markdown
# pkh
|
|
|
|
`pkh` is a packaging helper for Debian/Ubuntu packages.
|
|
|
|
|
|
|
|
## Installation
|
|
|
|
From crates.io:
|
|
|
|
```
|
|
cargo install pkh
|
|
```
|
|
|
|
Or build from source (the same system packages are needed either way):
|
|
|
|
```
|
|
sudo apt install pkg-config libssl-dev libgpg-error-dev libgpgme-dev
|
|
git clone https://git.vhaudiquet.fr/vhaudiquet/pkh.git
|
|
cd pkh
|
|
cargo install --path .
|
|
```
|
|
|
|
At runtime pkh shells out to the Debian packaging toolchain (git,
|
|
dpkg-dev, quilt, mmdebstrap, lintian, pristine-tar, autopkgtest, ...):
|
|
install the ones your workflows use, or build the classic snap from
|
|
`snap/snapcraft.yaml` (`snapcraft pack`), which carries them.
|
|
|
|
## Usage and features
|
|
|
|
### Basic concepts
|
|
|
|
`pkh` aims at wrapping the different debian tools and workflows
|
|
(`git`, `git-ubuntu`, `pull-debian-sources`, `pull-lp-sources`, `pull-ppa-sources`,
|
|
`dch`, `dpkg-buildpackage`, `sbuild`, `dpkg-source`, `quilt`, ...) into one tool, that
|
|
would have the same interface for everything, while being smarter at integrating all workflows.
|
|
|
|
Thus, `pkh` uses similar options for all subcommands (with very few command-specific options):
|
|
```
|
|
Options:
|
|
-s, --series <series> Target package distribution series
|
|
-d, --dist <dist> Target package distribution (debian, ubuntu)
|
|
-v, --version <version> Target package version
|
|
-a, --arch <arch> Target architecture (amd64, arm64, riscv64, ...)
|
|
-p, --pocket <pocket> Target distribution pocket (updates, security, proposed, ...)
|
|
--ppa <ppa> Do the action in/for a specific PPA
|
|
```
|
|
|
|
Commands and workflows include:
|
|
```
|
|
Commands:
|
|
new Scaffold a new Debian source package (buildable right away)
|
|
pull Pull a source package from the archive or git
|
|
chlog Auto-generate changelog entry, editing it, committing it afterwards
|
|
build Build the source package (into a .dsc)
|
|
put Upload the built source package to a PPA
|
|
deb Build the source package into binary package (.deb)
|
|
test Run the package's DEP-8 as-installed tests (autopkgtests)
|
|
lint Lint the package (lintian wrapper + pkh-native checks)
|
|
prune Prune residual pkh build artifacts and caches
|
|
help Print this message or the help of the given subcommand(s)
|
|
|
|
Options:
|
|
-h, --help Print help
|
|
-V, --version Print version
|
|
```
|
|
|
|
### Examples
|
|
|
|
A typical workflow to patch an Ubuntu package `hello` on the development release could be:
|
|
```
|
|
# Obtain the package source
|
|
git ubuntu clone hello
|
|
cd hello
|
|
# Apply the patch to the package
|
|
...
|
|
dpkg-source --commit
|
|
# Increment version number
|
|
dch
|
|
# Test that the patch builds
|
|
git ubuntu export-orig
|
|
dpkg-buildpackage -S -I -i -nc -d
|
|
sbuild --dist resolute --arch amd64 ../hello_xxx.dsc
|
|
# Upload the package to a ppa
|
|
dput ppa:user/hello_xxx ../hello_xxx_source.changes
|
|
# Commit the changes to git
|
|
git add debian/patches/xxx.patch
|
|
git commit -m "Applied patch xxx"
|
|
git add debian/changelog
|
|
git commit -m "changelog"
|
|
git checkout -b xxx
|
|
git push xxx user-fork
|
|
```
|
|
|
|
That is a lot of different tools and operations. With pkh, the same workflow:
|
|
```
|
|
# Obtain the package source (and orig tarball)
|
|
pkh pull hello # needs -d ubuntu if you are not running Ubuntu
|
|
# Apply the patch to the package
|
|
...
|
|
git add debian/patches/xxx.patch
|
|
git commit -m "Applied patch xxx"
|
|
pkh chlog
|
|
git add debian/changelog
|
|
git commit -m "d/changelog"
|
|
# Test that the package builds
|
|
pkh build
|
|
pkh deb
|
|
# Upload the package to a ppa
|
|
pkh put --ppa user/hello_xxx
|
|
# Push the commits to your fork
|
|
git push xxx user-fork
|
|
```
|
|
|
|
### Incremental builds (build sessions)
|
|
|
|
Every `pkh deb` build records a **session** under `/var/tmp/pkh/sessions`:
|
|
the bootstrapped chroot, the installed build dependencies and the build
|
|
artifacts of the staged tree. When a build fails (or is interrupted), the
|
|
session is kept and can be resumed:
|
|
|
|
```
|
|
pkh deb # fails after 25 minutes
|
|
pkh deb --resume # reuses the chroot, build deps and objects;
|
|
# only what changed is recompiled
|
|
pkh deb list # the sessions of this tree, with their ids
|
|
pkh deb --resume <id> # resume a specific session
|
|
pkh deb --keep # keep the session even after a successful build
|
|
# (iterate: edit, `pkh deb --resume --keep`, ...)
|
|
pkh prune # garbage-collect old sessions (7-day retention)
|
|
```
|
|
|
|
A plain `pkh deb` never reuses a session — everything is rechecked from
|
|
scratch — and it replaces the session of its target. `pkh deb --resume`
|
|
refuses to adopt a session built for a different series/architecture.
|
|
|
|
Packages whose `debian/rules` keeps an input-untracked stamp cache (the
|
|
kernels' `debian/stamps`) declare it in the quirks data (`resume_clear`):
|
|
resumed builds drop the cache so the wrapper steps — including the
|
|
kernel's flavour config export — re-run over the incremental inner
|
|
builds, and a config edit is never silently ignored. If a build died
|
|
hard (OOM kill), the next build unmounts and clears the leftover session
|
|
state first (escalating through sudo when needed).
|
|
|
|
### Running the DEP-8 tests
|
|
|
|
`pkh test` runs the package's as-installed tests (`debian/tests/control`,
|
|
declared by `Testsuite: autopkgtest` in `debian/control`) through a
|
|
native runner on a pkh testbed, and reports one verdict per test:
|
|
|
|
```
|
|
pkh test # test this tree: builds it first if needed
|
|
pkh test --mode local # on the current context, no isolation
|
|
pkh test --mode vm --image img.qcow2 # in a VM, via autopkgtest
|
|
pkh test --json --fail-on skip # CI-shaped
|
|
pkh test --list-tests # what does this package test?
|
|
```
|
|
|
|
By default the tests run in an ephemeral unshare chroot bootstrapped
|
|
from the same cached tarballs as `pkh deb`, against the binaries pkh deb
|
|
just built next to the tree (stale or missing output is rebuilt first;
|
|
`--no-build` refuses, `--debs` overrides). Pockets and PPAs resolve test
|
|
dependencies like in `pkh deb` (`-p`, `--ppa`), and `--test-name`,
|
|
`--skip-test`, `--shell-fail` and `--setup-commands` cover the day-to-day
|
|
debugging loop. Tests needing a real machine (`isolation-machine`,
|
|
`needs-reboot`) run only in `--mode vm`, which execs the installed
|
|
`autopkgtest` with its qemu runner.
|
|
|
|
Exit codes: 0 all tests passed (skips and flaky allowed), 1 at least one
|
|
test failed (or a `--fail-on` trigger), 2 runtime errors (no declared
|
|
tests, no usable testbed, ...).
|
|
|
|
## Future improvement ideas
|
|
|
|
- pull: try to fetch the correct git branch for series on Debian
|
|
- deb: asynchronous build, detachable and monitorable
|
|
- put: allow uploads to Debian or Ubuntu archives
|
|
- pull: cache Sources.gz files to improve speed
|
|
- pull: 'pkh pull' in a package tree should git pull and re-fetch orig tgz
|