vhaudiquet
81f0d52f85
The single mutable session per identity loses the last-good state: a failed run after a good one adopts the session in place and leaves only the failed state behind, so "continue from the last success" was impossible. Each build now records an immutable record under /var/tmp/pkh/sessions/<slug>/<id>/ (chroot, tree-upper, journal, manifest), and --incremental forks a NEW record — a full copy of the base's chroot and, at the same changelog version, its staged artifacts — under the base's flock, never mutating the base itself. Selection picks the newest successful record of the identity among the last five usable, falling back to the newest when none succeeded, and --incremental <id> pins an exact record (failed ones allowed: their environment may be fine). The lifecycle is strict keep-iff-incremental: plain runs build in a throwaway record discarded on success, failure and interrupt alike (the SIGINT cleanup hook follows the same policy), while --incremental records are always kept. Each identity keeps its newest five records; pkh prune's 7-day sweep and orphaned-lock cleanup follow the nested layout. --resume and --keep are removed (no aliases): their adoption-in-place and keep-on-success semantics are exactly what the record model replaces. pkh deb list shows id, outcome, tree version, fork lineage and age. README rewritten around the record workflow.
199 lines
7.5 KiB
Markdown
199 lines
7.5 KiB
Markdown
# pkh
|
|
|
|
`pkh` is a packaging helper for Debian/Ubuntu packages.
|
|
|
|
|
|
|
|
## Installation
|
|
|
|
From crates.io:
|
|
|
|
```
|
|
cargo install pkh
|
|
```
|
|
|
|
Or build from source (the same system packages are needed either way):
|
|
|
|
```
|
|
sudo apt install pkg-config libssl-dev libgpg-error-dev libgpgme-dev
|
|
git clone https://git.vhaudiquet.fr/vhaudiquet/pkh.git
|
|
cd pkh
|
|
cargo install --path .
|
|
```
|
|
|
|
At runtime pkh shells out to the Debian packaging toolchain (git,
|
|
dpkg-dev, quilt, mmdebstrap, lintian, pristine-tar, autopkgtest, ...):
|
|
install the ones your workflows use, or build the classic snap from
|
|
`snap/snapcraft.yaml` (`snapcraft pack`), which carries them.
|
|
|
|
## Usage and features
|
|
|
|
### Basic concepts
|
|
|
|
`pkh` aims at wrapping the different debian tools and workflows
|
|
(`git`, `git-ubuntu`, `pull-debian-sources`, `pull-lp-sources`, `pull-ppa-sources`,
|
|
`dch`, `dpkg-buildpackage`, `sbuild`, `dpkg-source`, `quilt`, ...) into one tool, that
|
|
would have the same interface for everything, while being smarter at integrating all workflows.
|
|
|
|
Thus, `pkh` uses similar options for all subcommands (with very few command-specific options):
|
|
```
|
|
Options:
|
|
-s, --series <series> Target package distribution series
|
|
-d, --dist <dist> Target package distribution (debian, ubuntu)
|
|
-v, --version <version> Target package version
|
|
-a, --arch <arch> Target architecture (amd64, arm64, riscv64, ...)
|
|
-p, --pocket <pocket> Target distribution pocket (updates, security, proposed, ...)
|
|
--ppa <ppa> Do the action in/for a specific PPA
|
|
```
|
|
|
|
Commands and workflows include:
|
|
```
|
|
Commands:
|
|
new Scaffold a new Debian source package (buildable right away)
|
|
pull Pull a source package from the archive or git
|
|
chlog Auto-generate changelog entry, editing it, committing it afterwards
|
|
build Build the source package (into a .dsc)
|
|
put Upload the built source package to a PPA
|
|
deb Build the source package into binary package (.deb)
|
|
test Run the package's DEP-8 as-installed tests (autopkgtests)
|
|
lint Lint the package (lintian wrapper + pkh-native checks)
|
|
prune Prune residual pkh build artifacts and caches
|
|
help Print this message or the help of the given subcommand(s)
|
|
|
|
Options:
|
|
-h, --help Print help
|
|
-V, --version Print version
|
|
```
|
|
|
|
### Examples
|
|
|
|
A typical workflow to patch an Ubuntu package `hello` on the development release could be:
|
|
```
|
|
# Obtain the package source
|
|
git ubuntu clone hello
|
|
cd hello
|
|
# Apply the patch to the package
|
|
...
|
|
dpkg-source --commit
|
|
# Increment version number
|
|
dch
|
|
# Test that the patch builds
|
|
git ubuntu export-orig
|
|
dpkg-buildpackage -S -I -i -nc -d
|
|
sbuild --dist resolute --arch amd64 ../hello_xxx.dsc
|
|
# Upload the package to a ppa
|
|
dput ppa:user/hello_xxx ../hello_xxx_source.changes
|
|
# Commit the changes to git
|
|
git add debian/patches/xxx.patch
|
|
git commit -m "Applied patch xxx"
|
|
git add debian/changelog
|
|
git commit -m "changelog"
|
|
git checkout -b xxx
|
|
git push xxx user-fork
|
|
```
|
|
|
|
That is a lot of different tools and operations. With pkh, the same workflow:
|
|
```
|
|
# Obtain the package source (and orig tarball)
|
|
pkh pull hello # needs -d ubuntu if you are not running Ubuntu
|
|
# Apply the patch to the package
|
|
...
|
|
git add debian/patches/xxx.patch
|
|
git commit -m "Applied patch xxx"
|
|
pkh chlog
|
|
git add debian/changelog
|
|
git commit -m "d/changelog"
|
|
# Test that the package builds
|
|
pkh build
|
|
pkh deb
|
|
# Upload the package to a ppa
|
|
pkh put --ppa user/hello_xxx
|
|
# Push the commits to your fork
|
|
git push xxx user-fork
|
|
```
|
|
|
|
`pkh chlog` edits `debian/changelog` by default; point it at another file
|
|
with `--file <path>` (`-f`).
|
|
|
|
`pkh chlog` walks the commit history back to the previous changelog entry.
|
|
Without a version tag, that search is capped at the last 500 commits
|
|
(`--limit <n>`, `0` = unlimited): beyond the cap, the entry simply starts
|
|
at the oldest commit searched, which keeps the run fast on large histories.
|
|
|
|
### Incremental builds (build records)
|
|
|
|
Every `pkh deb` build runs in its own immutable **record** under
|
|
`/var/tmp/pkh/sessions/<target>/<id>/`: the bootstrapped chroot, the
|
|
installed build dependencies and the build artifacts of the staged
|
|
tree. `--incremental` forks a previous record — a full copy, the base
|
|
is never modified — so a failed run can never destroy the last-good
|
|
environment it was built from:
|
|
|
|
```
|
|
pkh deb # fully ephemeral: nothing is kept, even on
|
|
# failure or interrupt
|
|
pkh deb --incremental # fork the newest successful record of the
|
|
# target (last five usable): the chroot, build
|
|
# deps and, at the same version, the objects
|
|
# are reused; only what changed is recompiled.
|
|
# The result is always kept (success or
|
|
# failure) for the next fork
|
|
pkh deb --incremental <id>
|
|
# fork that exact record (see `pkh deb list`);
|
|
# a failed record is fine if its environment is
|
|
pkh deb list # the records of this tree: id, outcome,
|
|
# version, fork lineage, age
|
|
pkh prune # garbage-collect records (7-day retention)
|
|
```
|
|
|
|
Each target (series/arch) keeps its last five records — older ones are
|
|
pruned as new ones are created, and `pkh prune` removes those untouched
|
|
for a week. `pkh deb --incremental` never forks a record built for a
|
|
different series/architecture, and a plain `pkh deb` still never
|
|
reuses anything: it builds in a throwaway record that is discarded on
|
|
success, failure and interrupt alike.
|
|
|
|
Packages whose `debian/rules` keeps an input-untracked stamp cache (the
|
|
kernels' `debian/stamps`) declare it in the quirks data
|
|
(`resume_clear`): forked builds drop the cache so the wrapper steps —
|
|
including the kernel's flavour config export — re-run over the
|
|
incremental inner builds, and a config edit is never silently ignored.
|
|
If a build died hard (OOM kill), the next fork unmounts the leftover
|
|
state first (escalating through sudo when needed).
|
|
|
|
### Running the DEP-8 tests
|
|
|
|
`pkh test` runs the package's as-installed tests (`debian/tests/control`,
|
|
declared by `Testsuite: autopkgtest` in `debian/control`) through a
|
|
native runner on a pkh testbed, and reports one verdict per test:
|
|
|
|
```
|
|
pkh test # test this tree: builds it first if needed
|
|
pkh test --mode local # on the current context, no isolation
|
|
pkh test --mode vm --image img.qcow2 # in a VM, via autopkgtest
|
|
pkh test --json --fail-on skip # CI-shaped
|
|
pkh test --list-tests # what does this package test?
|
|
```
|
|
|
|
By default the tests run in an ephemeral unshare chroot bootstrapped
|
|
from the same cached tarballs as `pkh deb`, against the binaries pkh deb
|
|
just built next to the tree (stale or missing output is rebuilt first;
|
|
`--no-build` refuses, `--debs` overrides). Pockets and PPAs resolve test
|
|
dependencies like in `pkh deb` (`-p`, `--ppa`), and `--test-name`,
|
|
`--skip-test`, `--shell-fail` and `--setup-commands` cover the day-to-day
|
|
debugging loop. Tests needing a real machine (`isolation-machine`,
|
|
`needs-reboot`) run only in `--mode vm`, which execs the installed
|
|
`autopkgtest` with its qemu runner.
|
|
|
|
Exit codes: 0 all tests passed (skips and flaky allowed), 1 at least one
|
|
test failed (or a `--fail-on` trigger), 2 runtime errors (no declared
|
|
tests, no usable testbed, ...).
|
|
|
|
## Future improvement ideas
|
|
|
|
- pull: try to fetch the correct git branch for series on Debian
|
|
- deb: asynchronous build, detachable and monitorable
|
|
- put: allow uploads to Debian or Ubuntu archives
|
|
- pull: cache Sources.gz files to improve speed
|
|
- pull: 'pkh pull' in a package tree should git pull and re-fetch orig tgz
|