Commit Graph
117 Commits
Author SHA1 Message Date
vhaudiquet 6a5c5a7106 context: fix deadlock and panics when resolving parented contexts
set_current held the config RwLock for writing across make_context,
which for a context with a parent re-entered the same lock through
Context::new's global-manager lookup, deadlocking 'pkh context use'.
Context building is now lock-free by construction: make_context
resolves parent chains against a snapshot map (also rejecting parent
cycles), and neither set_current nor remove_context holds a guard
while building a Context.

A corrupt contexts.json no longer aborts every command at manager
init: load falls back to the default local-only config, backs the
corrupt file up to contexts.json.bak so a later save cannot silently
destroy it, and a dangling current/parent context falls back to local
with an error log instead of panicking.
2026-09-16 02:27:11 +02:00
vhaudiquet 592e98c1e9 deb: pass the build context explicitly instead of swapping the global
build_binary_package installed its ephemeral chroot context into the
process-global manager and read it back with context::current(),
ignoring its ctx parameter: two concurrent builds would re-point each
other's global and each drop would clean up whichever chroot was
current at the time. The guard now keeps the Arc of the context it
created (parented directly on the base context, not on a config-name
lookup), exposes it via context(), and Drop cleans up exactly that
context and restores the exact handle that was current at creation,
so overlapping builds no longer cross-destroy each other.
2026-09-16 02:05:40 +02:00
vhaudiquet 512a1cb778 deb: unmount and remove the ephemeral chroot on Ctrl-C
The SIGINT handler libc::_exit(130)s, skipping EphemeralContextGuard's
drop and leaking the freshly bootstrapped chroot with its bind-mounted
/proc and overlay mounts. Resources now register a self-contained
cleanup hook in a process-global registry that the handler drains right
before exiting: the hook unmounts everything under the chroot path
(children first, lazy fallback) and removes the tree, using only stored
paths and direct umount/rm subprocesses so it cannot deadlock on a lock
the interrupted thread may hold; sudo -n keeps it from ever hanging on
a password prompt. Drop deregisters the hook first, so the normal
cleanup path is unchanged.

Also fixes the hex grouping of the CRC-24 polynomial in apt::release.
2026-09-16 01:55:35 +02:00
vhaudiquet 685538e637 apt: round-trip sources in place instead of consolidating them into sources.list
Saving the modified source entries with save_legacy rewrote every entry
into /etc/apt/sources.list in legacy format, destroying deb-src entries
and signed-by/trusted options, duplicating every distro entry that came
from a deb822 file (which stayed in place), and hardcoding the Ubuntu
keyring on cross builds. Entries now remember the file and format they
were loaded from and are written back there; new entries (PPAs, ports)
go to a pkh-owned /etc/apt/sources.list.d/pkh-added.list, and a one-time
<path>.pkh-backup copy is made before overwriting an existing file.

Also fixes: 'Types: deb deb-src' stanzas are split instead of being
treated as binary-only, commented-out legacy entries are kept disabled
instead of deleted, debian.sources is actually read on Debian (the old
else-if never fired), and the double blank lines save_legacy emitted.
2026-09-16 00:10:45 +02:00
vhaudiquet b34e86dcfe deb: scope the arch-indep build-dep pass to the host arch in cross builds
CI / build (push) Successful in 2m49s
CI / test (push) Skipped
CI / snap (push) Failing after 11s
Without --host-architecture, the second build-dep pass re-resolves the
whole Build-Depends field for the native architecture: apt swaps
host-arch -dev packages for native ones (e.g. libcurl4-gnutls-dev,
whose arch-differing curl-config makes dpkg refuse the co-install) and
breaks the cross build environment.

Per dpkg-checkbuilddeps, both Build-Depends and Build-Depends-Indep
resolve for the host architecture in cross mode, so pass
--host-architecture to the second pass as well. Skip the pass entirely
when the source declares no Build-Depends-Indep.

Add an end-to-end regression test building a package that declares
libdb-dev in both fields and links a host-arch binary against it: the
test only passes if the arch-indep pass did not swap the arm64 -dev
packages for native ones.
2026-09-15 18:05:57 +02:00
vhaudiquet 5500f98586 pull: add --repository to pull from external flat repositories
Add a --repository flag taking the full suite URL of an external flat
repository (e.g. https://pkg.noctalia.dev/deb/resolute/), i.e. one with
no dists/ hierarchy, like apt's exact-path suites ('Suites: resolute/').

The suite name is read from the root Release file (Codename/Suite), the
sources index is fetched from the repository root as Sources.xz/gz/plain,
and package files are resolved against the URL root, ignoring the stanza
Directory field like apt does. As with PPAs, the stanza Vcs-Git is never
used for external repositories, so the source always comes from the
repository itself.

Also make the sources index parser detect compression by magic bytes
(gz/xz/plain) instead of assuming gzip, and fix extraction of archives
with './'-prefixed entries, which previously aborted and are now
extracted in place instead of being relocated.
2026-09-15 10:57:06 +02:00
vhaudiquet 3b99ece39a fmt
CI / build (push) Successful in 2m46s
CI / test (push) Skipped
CI / snap (push) Failing after 28s
2026-09-10 15:31:17 +02:00
vhaudiquet c7af3bc9b1 deb: retrieve only the artifacts produced by the build, not globbed files
CI / build (push) Failing after 53s
CI / test (push) Skipped
CI / snap (push) Skipped
build_binary_package_impl copied the whole parent directory into the
build root (ensure_available) and then retrieved every *.deb / *.changes
/ *.buildinfo it found there. That surfaced stale files already sitting
next to the package tree in the "Built in Ns:" summary.

Now local::build returns the exact set of artifacts produced by this
build — the binary packages registered in debian/files plus the
generated .buildinfo/.changes from generate_binary_metadata — and
build_binary_package_impl retrieves that list instead of globbing the
build root. Only files genuinely produced by the current build are
printed.
2026-09-10 15:30:06 +02:00
vhaudiquet 182a06ffbe deb: add -j/--jobs to control parallel build jobs
By default the number of parallel jobs is detected with nproc inside
the build context. Add a -j/--jobs option so an explicit count can be
honored instead, threading it through build_binary_package and
local::build into DEB_BUILD_OPTIONS=parallel=N.
2026-09-10 15:27:50 +02:00
vhaudiquet 429429e414 fmt 2026-08-24 11:58:19 +02:00
vhaudiquet 42fcfc2dfa build/binary: native .buildinfo/.changes for binary builds (pkh deb)
Extend the metadata writers to binary-only uploads and wire them into
the 'pkh deb' flow:

- build/binary.rs generates <pkg>_<ver>_<arch>.buildinfo/.changes
  through any Context: debian/files consumption, encounter-order
  Architecture accumulation (sorted in .buildinfo like dpkg-genbuildinfo),
  sorted Binary lists, dpkg-formatted Description lines with udeb
  suffixes, Installed-Build-Depends closure over the context status DB,
  and binNMU handling (Source: pkg (prev), Binary-Only-Changes, previous
  .dsc redistribution);
- artifact digests are computed inside the context via coreutils
  (md5sum/sha1sum/sha256sum/stat) so chrooted/remote trees work;
- deb/local.rs runs the generation after 'rules binary', exports
  SOURCE_DATE_EPOCH from the changelog (reproducibility), and resolves
  vendor/profiles inside the context; deb/mod.rs retrieves the new
  artifacts alongside the debs;
- reusable helpers added: FilesList::parse/render,
  parse_changelog_entry_from_str, parse_previous_version_from_str,
  installed_build_depends_from_content.

Differential gate: same tree built with real 'dpkg-buildpackage -b' and
with the pkh flow; .changes/.buildinfo compared field-by-field modulo
machine-dependent fields, artifact checksums included.
2026-08-24 11:58:19 +02:00
vhaudiquet e5adf600c3 deb: change ui/ux of pkh deb
CI / build (push) Successful in 2m54s
CI / test (push) Skipped
CI / snap (push) Failing after 12s
2026-08-22 22:26:20 +02:00
vhaudiquet 18be62d4a0 clippy
CI / build (push) Failing after 32m38s
CI / snap (push) Skipped
2026-08-22 12:46:22 +02:00
vhaudiquet 24faad6e11 deb: pin proposed pocket when building against it
'proposed' pockets are marked 'NotAutomatic' in their Release file,
which gives them an apt priority of 1: apt would ignore them during
build-dependency resolution even when the suite is enabled.

Write an apt preferences entry pinning '{series}-proposed' at
priority 600 so build-dependencies are actually resolved from the
requested pocket.

Apt preferences are global: a single 'release' pin matches the pinned
suite on every repository carrying it (archive, security and ports),
for all architectures, so this also covers cross-builds pulling
dependencies from 'ports.ubuntu.com'.
2026-08-22 12:45:16 +02:00
vhaudiquet cc7cbaaa8f deb: apply quilt patches before building 2026-08-22 12:31:30 +02:00
vhaudiquet f54be8ad8c pull,deb: add top-level --pocket option
CI / build (push) Failing after 2m31s
CI / snap (push) Skipped
- pull: select the pocket to download the source package from
- deb: build against dependencies from a specific pocket:
  - local mode: enable '{series}-{pocket}' suite on archive sources
  - cross mode: include the pocket suite in required repositories
2026-08-22 12:02:05 +02:00
vhaudiquet 317dbcd062 deb: use overlayfs instead of copy
CI / build (push) Failing after 14m44s
CI / snap (push) Has been skipped
2026-08-12 15:44:13 +02:00
vhaudiquet e3d855ca66 deb: detect package directory using cwd
CI / build (push) Successful in 16m19s
CI / snap (push) Successful in 3m52s
remove linux-riscv specific quirk
2026-08-11 11:33:12 +02:00
vhaudiquet 62ce4e3696 deb: remove sbuild build mode
CI / build (push) Successful in 8m15s
CI / snap (push) Successful in 3m28s
2026-07-24 14:32:47 +02:00
vhaudiquet f9e11e951b clippy
CI / build (push) Failing after 8m37s
CI / snap (push) Has been skipped
2026-07-24 10:53:27 +02:00
vhaudiquet 768e1c4f78 fmt
CI / build (push) Failing after 1m56s
CI / snap (push) Has been skipped
2026-07-22 14:36:28 +02:00
vhaudiquet 48248fdf9c Merge branch 'main' of https://git.vhaudiquet.fr/vhaudiquet/pkh
CI / build (push) Failing after 1m1s
CI / snap (push) Has been skipped
* 'main' of https://git.vhaudiquet.fr/vhaudiquet/pkh:
  deb: fix injection of .deb packages with apt-get
  deb: allow injecting .deb files packages
2026-07-17 10:12:48 +02:00
vhaudiquet c4b59a4376 context: copy symlinks as symlinks and improve error messages 2026-07-17 10:12:46 +02:00
vhaudiquet 73511c258b deb: fix injection of .deb packages with apt-get
CI / build (push) Successful in 9m5s
CI / snap (push) Successful in 3m56s
2026-04-04 12:41:52 +02:00
vhaudiquet d83174c980 deb: allow injecting .deb files packages
CI / build (push) Failing after 1m19s
CI / snap (push) Has been skipped
2026-04-04 12:35:14 +02:00
vhaudiquet 2b27b7b06e deb: add quirks to allow packages to declare specific package directories
CI / build (push) Successful in 8m41s
CI / snap (push) Successful in 3m55s
2026-03-25 17:40:55 +01:00
vhaudiquet 32e15b1106 deb: ignore linux-riscv test
CI / build (push) Successful in 8m16s
CI / snap (push) Failing after 7s
2026-03-19 11:27:02 +01:00
vhaudiquet 7640952bdc unshare: mount proc differently depending on root privileges 2026-03-19 11:26:10 +01:00
vhaudiquet 2b6207981a deb: make tests parallel
CI / build (push) Failing after 12m10s
CI / snap (push) Has been skipped
2026-03-19 00:24:48 +01:00
vhaudiquet daaf33cd6b deb: fix race condition for test
CI / build (push) Failing after 18m26s
CI / snap (push) Has been skipped
Fix race condition around current context,
related to find_package_directory.
2026-03-18 23:34:33 +01:00
vhaudiquet 4a73e6e1d6 deb: fix race condition for tests
CI / build (push) Failing after 20m7s
CI / snap (push) Has been skipped
2026-03-18 17:35:38 +01:00
vhaudiquet d06e091121 apt/keyring: download 3 keyrings for sid
CI / build (push) Failing after 21m24s
CI / snap (push) Has been skipped
2026-03-18 15:23:57 +01:00
vhaudiquet dce39c9a84 deb: allow adding multiple ppas for deps
CI / build (push) Failing after 13m29s
CI / snap (push) Has been skipped
2026-02-19 15:36:08 +01:00
vhaudiquet 02dbb41219 deb: fix bugs on injecting packages/ppa
- ppa was not the right target_arch if binfmt was used
- apt-get would not downgrade for injected packages, but we want to
2026-02-19 15:19:02 +01:00
vhaudiquet b508dd3d82 deb: download keyring in cache directory for mmdebstrap
CI / build (push) Failing after 13m33s
CI / snap (push) Has been skipped
also avoid sudo if we are root
2026-02-19 11:30:33 +01:00
vhaudiquet 87b48bf9c8 deb: add --inject to inject a package
CI / build (push) Failing after 20m25s
CI / snap (push) Has been skipped
2026-02-18 23:19:57 +01:00
vhaudiquet 7c53b268dd deb: use qemu_binfmt on ephemeral/local builds of different arch
CI / build (push) Failing after 18m19s
CI / snap (push) Has been skipped
2026-02-11 11:58:59 +01:00
vhaudiquet 97725efb34 Multiple changes
CI / build (push) Failing after 18m0s
CI / snap (push) Has been skipped
- quirks: added quirks, that does nothing for now
- deb: install arch-independant dependencies (too much is better)
- deb: added linux-riscv crossbuild test
2026-02-10 11:07:05 +01:00
vhaudiquet 8345f51d2f Multiple changes:
CI / build (push) Successful in 16m23s
CI / snap (push) Successful in 4m1s
- deb: can use ppa as dependency
- deb: cross and regular are using parallel nocheck builds
- deb: ephemeral will not pull keyring if no root powers
2026-02-06 12:04:25 +01:00
vhaudiquet 225157be63 pull: allow pulling from ppa
CI / build (push) Successful in 15m54s
CI / snap (push) Successful in 3m19s
2026-01-29 17:11:01 +01:00
vhaudiquet af73c14674 deb: fix package directory detection by looking for 'debian' subfolder
CI / build (push) Successful in 16m35s
CI / snap (push) Successful in 3m35s
2026-01-28 15:59:31 +01:00
vhaudiquet c64c3146d7 deb: test building packages from sid
CI / build (push) Failing after 9m17s
2026-01-26 12:06:00 +01:00
vhaudiquet 13c44daf9a deb: allow use parallel building
CI / build (push) Successful in 10m29s
2026-01-26 10:31:58 +01:00
vhaudiquet 73a61042e8 deb: disable gcc-15 test for CI 2026-01-26 10:25:53 +01:00
vhaudiquet 2f5fb3b1e7 deb: fix runtime usage for chroot
CI / build (push) Failing after 11m13s
2026-01-26 10:14:04 +01:00
vhaudiquet dd9cc07285 deb: make sure to have the right apt keyrings
CI / build (push) Failing after 14s
2026-01-20 19:31:07 +01:00
vhaudiquet a975c24e54 deb: ci test building gcc-15 from sid
CI / build (push) Failing after 9m17s
2026-01-14 11:12:12 +01:00
vhaudiquet b4a60e2ae2 deb: cleanup ephemeral context only on success
CI / build (push) Successful in 11m39s
2026-01-13 14:54:56 +01:00
vhaudiquet 87ce0f648b deb: create special device nodes inside chroot
CI / build (push) Successful in 9m5s
2026-01-13 09:58:16 +01:00
vhaudiquet 593793373a deb: consider all kinds of package dirs
CI / build (push) Failing after 12m32s
2026-01-12 11:29:18 +01:00