From ddbf91108f310a753c6bd00ae6ce406ec58c50d0 Mon Sep 17 00:00:00 2001 From: Mathieu Benoit Date: Mon, 17 Aug 2026 01:33:41 -0400 Subject: [PATCH] [FIX] libdir: the Debian host hid the libraries from Arch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit meson and cmake both ask the HOST where libraries go. On Ubuntu the answer is lib/s390x-linux-gnu, so five packages already in the repository ship theirs where Arch's ld.so and pkgconf never look. Measured: expat 12 lz4 6 pacman 6 pkgconf 6 zstd 12 entries Nothing failed. util-linux is where it finally shouted, and even there the rm was the first victim, not the cause. pacman is the one that matters: libalpm.so.16 landed where pacman's own binary cannot load it, and a target installed from that package has no working package manager left to repair itself with. arch-meson now states the libdir devtools has no need to state, and is reinstalled on every run -- editing the copy here changed nothing while /usr/local/bin held a stale one. Four packages call bare meson or cmake and get their own hook. util-linux's old hook chased lib64, which never existed here; it is deleted. --- FR --- meson et cmake demandent tous deux à l'HÔTE où vont les bibliothèques. Sous Ubuntu la réponse est lib/s390x-linux-gnu : cinq paquets déjà dans le dépôt livrent donc les leurs là où ld.so et pkgconf d'Arch ne regarderont jamais. Mesuré : expat 12 lz4 6 pacman 6 pkgconf 6 zstd 12 entrées Rien n'a échoué. util-linux est l'endroit où cela a fini par crier, et même là le rm était la première victime, pas la cause. pacman est celui qui compte : libalpm.so.16 atterrissait là où le binaire de pacman ne peut pas la charger, et une cible installée depuis ce paquet n'a plus de gestionnaire de paquets pour se réparer. arch-meson énonce désormais le libdir que devtools n'a pas besoin d'énoncer, et se réinstalle à chaque exécution : éditer la copie du dépôt ne changeait rien tant que /usr/local/bin en gardait une périmée. Quatre paquets appellent meson ou cmake nu et reçoivent leur crochet. L'ancien crochet util-linux poursuivait un lib64 qui n'a jamais existé ici : il est supprimé. Assisted-by: Claude Opus 5 --- patches/pkgbuild/expat.sh | 43 ++++++++++++++++++++++++++ patches/pkgbuild/lz4.sh | 41 +++++++++++++++++++++++++ patches/pkgbuild/pacman.sh | 55 ++++++++++++++++++++++++++++++++++ patches/pkgbuild/util-linux.sh | 34 --------------------- patches/pkgbuild/zstd.sh | 52 ++++++++++++++++++++++++++++++++ scripts/bootstrap-pacman.sh | 15 ++++++++++ scripts/build-stage1.sh | 4 +++ scripts/devtools/arch-meson | 38 +++++++++++++++++++++++ 8 files changed, 248 insertions(+), 34 deletions(-) create mode 100755 patches/pkgbuild/expat.sh create mode 100755 patches/pkgbuild/lz4.sh create mode 100755 patches/pkgbuild/pacman.sh delete mode 100755 patches/pkgbuild/util-linux.sh create mode 100755 patches/pkgbuild/zstd.sh diff --git a/patches/pkgbuild/expat.sh b/patches/pkgbuild/expat.sh new file mode 100755 index 0000000..7f45f91 --- /dev/null +++ b/patches/pkgbuild/expat.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +# expat: bare cmake picks up the Debian multiarch libdir. +# +# The PKGBUILD calls cmake directly -- NOT arch-cmake -- so the devtools +# stand-in that already passes -DCMAKE_INSTALL_LIBDIR=lib never runs. The +# option array only pins the prefix. +# +# expat's CMakeLists include(GNUInstallDirs), and GNUInstallDirs picks its +# default LIBDIR by sniffing the HOST: it sees /etc/debian_version, finds no +# /etc/arch-release, and sets lib/ = lib/s390x-linux-gnu. Because +# CMAKE_INSTALL_LIBDIR was never given, that default wins and is cached. +# +# The build then exits 0 and the package installs cleanly. It is the ARTEFACT +# that is wrong -- everything LIBDIR-derived lands off Arch's search paths: +# +# usr/lib/s390x-linux-gnu/libexpat.so.1.12.3 ld.so reads /usr/lib only +# usr/lib/s390x-linux-gnu/pkgconfig/expat.pc pkgconf reads /usr/lib only +# usr/lib/s390x-linux-gnu/cmake/expat-2.8.3/ find_package(EXPAT) misses +# +# and usr/lib itself holds nothing but that subdirectory. expat is a +# dependency of cmake, python, gdb and fontconfig, so this does not fail +# loudly here -- it fails later, in whatever links against -lexpat. +# +# The fix is to state the libdir the stand-in would have stated. Passing it on +# the command line makes it a cache entry BEFORE GNUInstallDirs runs, and +# GNUInstallDirs only fills in defaults for variables that are still unset. +# +# One configure call, so exactly one insertion: assert the count rather than +# rewriting every -D. +set -euo pipefail +python3 - <<'PY' +import io +s = io.open("PKGBUILD", encoding="utf-8").read() +assert "CMAKE_INSTALL_LIBDIR" not in s, "expat: libdir already set, hook ran twice" +anchor = " -D CMAKE_INSTALL_PREFIX=/usr\n" +n = s.count(anchor) +assert n == 1, "expat: expected 1 cmake option block, found %d" % n +s = s.replace(anchor, anchor + " -D CMAKE_INSTALL_LIBDIR=lib\n", 1) +io.open("PKGBUILD", "w", encoding="utf-8").write(s) +PY +[ "$(grep -c 'CMAKE_INSTALL_LIBDIR=lib' PKGBUILD)" = 1 ] || { + echo "expat: libdir not pinned exactly once" >&2; exit 1; } +echo "expat: CMAKE_INSTALL_LIBDIR=lib (bare cmake bypasses arch-cmake)" diff --git a/patches/pkgbuild/lz4.sh b/patches/pkgbuild/lz4.sh new file mode 100755 index 0000000..6048b78 --- /dev/null +++ b/patches/pkgbuild/lz4.sh @@ -0,0 +1,41 @@ +#!/usr/bin/env bash +# lz4: bare `meson setup`, so libdir lands on the Debian multiarch path. +# +# This PKGBUILD calls meson directly -- not arch-meson -- so nothing supplies +# --libdir and meson falls back to default_libdir(), which asks the HOST +# where libraries go. On this Debian-flavoured builder the answer comes from +# dpkg-architecture -qDEB_HOST_MULTIARCH, i.e. lib/s390x-linux-gnu. The +# package built, exited 0, and shipped: +# +# usr/lib/s390x-linux-gnu/liblz4.so.1.10.0 +# usr/lib/s390x-linux-gnu/pkgconfig/liblz4.pc +# +# Arch's ld.so and pkgconf only read /usr/lib, so on the target every consumer +# fails to link and `pkg-config liblz4` finds nothing -- while the PKGBUILD +# still declares provides=('liblz4.so'), a promise the artefact does not keep. +# The binaries in usr/bin work, which is exactly what makes it look fine. +# +# The damage is twofold: liblz4.pc is not merely in the wrong directory, its +# body also reads libdir=${prefix}/lib/s390x-linux-gnu, so relocating the file +# would not have been enough. Setting libdir fixes both at once. +# +# Anchor on "meson setup", NOT on "meson": build() also runs meson configure +# and meson compile, package() runs meson install, and check() runs +# build/meson/programs/lz4 -- a PATH containing the word. There is exactly one +# setup, asserted below. +# +# --libdir survives the later `meson configure`, which only rewrites the +# options it is handed (-Dcontrib -Dexamples -Dprograms). +set -euo pipefail +python3 - <<'PY' +import io +s = io.open("PKGBUILD", encoding="utf-8").read() +old = "meson setup --prefix=/usr --buildtype=plain" +n = s.count(old) +assert n == 1, "lz4: expected exactly 1 meson setup, found %d" % n +s = s.replace(old, "meson setup --prefix=/usr --libdir=lib --buildtype=plain", 1) +io.open("PKGBUILD", "w", encoding="utf-8").write(s) +PY +[ "$(grep -c -- '--libdir=lib' PKGBUILD)" = 1 ] || { + echo "lz4: --libdir=lib not applied exactly once" >&2; exit 1; } +echo "lz4: meson libdir pinned to lib (host default is lib/s390x-linux-gnu)" diff --git a/patches/pkgbuild/pacman.sh b/patches/pkgbuild/pacman.sh new file mode 100755 index 0000000..8c124a4 --- /dev/null +++ b/patches/pkgbuild/pacman.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +# pacman: bare meson takes its libdir from the Debian host. +# +# The build host is Ubuntu, so meson's default_libdir() asks +# dpkg-architecture for the multiarch tuple and hands back +# +# lib/s390x-linux-gnu +# +# devtools' arch-meson would override that, but this PKGBUILD calls BARE +# `meson`, so nothing does. The build then succeeds, makepkg exits 0, and the +# package ships +# +# usr/lib/s390x-linux-gnu/libalpm.so.16 +# +# where Arch's ld.so never looks -- it reads /usr/lib. libalpm is pacman's +# OWN library, so a target installed from this package has a package manager +# that cannot start, and no way to pacman -U the fix. Nothing in the build log +# says a word about it. This is the single most expensive instance of the +# libdir slip in the port, which is why it gets its own hook rather than +# waiting for the arch-meson stand-in to cover it. +# +# The same slip poisons two things downstream of the library file: +# - .../pkgconfig/libalpm.pc, whose `libdir=${prefix}/lib/s390x-linux-gnu` +# then aims every consumer at the same dead directory; +# - makepkg's own autodep. /etc/makepkg.conf sets +# LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32') and library_provides.sh runs +# `find "$pkgdir/$dir" -maxdepth 1`, so it never descends into the +# multiarch directory. .PKGINFO recorded a bare `provides = libalpm.so` +# with no version, where it should read libalpm.so=16-64 -- so every +# `libalpm.so=16-64` dependency in the port would go unsatisfiable, and +# the error would name the package that wanted it, never pacman. +# +# Only the configure call takes a libdir. The other four `meson` words in the +# file -- makedepends, compile, test, install -- take none and are left alone. +# The count is asserted rather than assumed. +# +# libdir only. sbindir and libexecdir also keep non-Arch defaults here, but +# pacman installs nothing into either -- the manifest is /usr/{bin,include, +# lib,share} and /etc, nothing else. Setting them would be a change with no +# artefact behind it, and if upstream ever does add an sbin binary we want to +# see it rather than have it silently absorbed. +set -euo pipefail +python3 - <<'PY' +import io +s = io.open("PKGBUILD", encoding="utf-8").read() +old = " meson --prefix=/usr \\\n" +n = s.count(old) +assert n == 1, "pacman: expected 1 meson configure call, found %d" % n +assert "libdir" not in s, "pacman: PKGBUILD already sets a libdir, re-read it" +s = s.replace(old, old + " --libdir=lib \\\n", 1) +io.open("PKGBUILD", "w", encoding="utf-8").write(s) +PY +[ "$(grep -c -- '--libdir=lib' PKGBUILD)" = 1 ] || { + echo "pacman: libdir not forced to lib exactly once" >&2; exit 1; } +echo "pacman: libdir forced to /usr/lib (Debian host defaults it to multiarch)" diff --git a/patches/pkgbuild/util-linux.sh b/patches/pkgbuild/util-linux.sh deleted file mode 100755 index 20b8eab..0000000 --- a/patches/pkgbuild/util-linux.sh +++ /dev/null @@ -1,34 +0,0 @@ -#!/usr/bin/env bash -# util-linux: s390x installs to lib64, Arch expects lib. -# -# rm: cannot remove '.../pkg/util-linux/usr/lib/lib*.la': No such file -# -# The same divergence already handled in gcc.sh: 64-bit Z installs libraries -# into usr/lib64, while every Arch PKGBUILD names usr/lib. Here it shows up -# as a glob that matches nothing, which `rm` reports as a missing file rather -# than an empty expansion -- so the error names a path that was never going -# to exist. -# -# Merged before package() touches anything, so the upstream paths keep -# working unchanged. -set -euo pipefail -python3 - <<'PY' -import io, re -s = io.open("PKGBUILD", encoding="utf-8").read() -merge = """ # s390x installs 64-bit libraries to lib64; Arch expects lib. - if [ -d "$pkgdir/usr/lib64" ]; then - mkdir -p "$pkgdir/usr/lib" - cp -a "$pkgdir/usr/lib64/." "$pkgdir/usr/lib/" - rm -rf "$pkgdir/usr/lib64" - fi -""" -m = re.search(r"\npackage_util-linux\(\) \{\n", s) -if not m: - m = re.search(r"\npackage\(\) \{\n", s) -assert m, "package function not found" -s = s[:m.end()] + merge + s[m.end():] -io.open("PKGBUILD", "w", encoding="utf-8").write(s) -PY -grep -q 'installs 64-bit libraries to lib64' PKGBUILD || { - echo "util-linux: lib64 merge not inserted" >&2; exit 1; } -echo "util-linux: lib64 merged into lib before packaging" diff --git a/patches/pkgbuild/zstd.sh b/patches/pkgbuild/zstd.sh new file mode 100755 index 0000000..53c58e4 --- /dev/null +++ b/patches/pkgbuild/zstd.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# zstd: bare cmake, so nobody passes -DCMAKE_INSTALL_LIBDIR. +# +# This PKGBUILD calls plain `cmake`, not the arch-cmake stand-in, so the +# -DCMAKE_INSTALL_LIBDIR=lib that stand-in supplies never arrives. zstd's own +# build/cmake/CMakeLists.txt does include(GNUInstallDirs), and GNUInstallDirs +# on this build host walks into its Debian branch: +# +# elseif (EXISTS "/etc/arch-release") <- absent here +# elseif (EXISTS "/etc/debian_version") <- present here +# +# and sets CMAKE_INSTALL_LIBDIR=lib/s390x-linux-gnu. The build then exits 0 +# and ships the library where Arch's ld.so and pkgconf will never look: +# +# usr/lib/s390x-linux-gnu/libzstd.so.1.5.7 +# usr/lib/ <- empty +# +# Note what is NOT the trap here. zstd looks like it might also run a plain +# `make` next to the cmake tree; it does not. There is exactly one configure +# (`cmake -S build/cmake -B build`), `cmake --build build` drives that same +# build dir, and `cmake --install build` installs from it. One insertion +# covers the whole package. +# +# It has to, because CMAKE_INSTALL_LIBDIR is the single root of four separate +# poisoned paths -- fixing only the .so would have left three: +# +# lib/CMakeLists.txt:279 LIBRARY DESTINATION -> libzstd.so* +# lib/CMakeLists.txt:270 pkgconfig -> libzstd.pc location +# lib/CMakeLists.txt:264 join_paths(LIBDIR) -> libdir= INSIDE libzstd.pc +# CMakeLists.txt:212 ConfigPackageLocation -> cmake/zstd/*.cmake +# +# The third is why this package is worse than it looks. The shipped +# libzstd.pc reads libdir=${exec_prefix}/lib/s390x-linux-gnu, so the poison +# does not stop at zstd: every downstream package that finds zstd through +# pkg-config inherits -L/usr/lib/s390x-linux-gnu and hardcodes a path that +# does not exist on the target. And zstd is a pacman dependency -- pacman +# links libzstd.so.1 -- so the current artefact yields a system that cannot +# run its own package manager to repair itself. +set -euo pipefail +python3 - <<'PY' +import io +s = io.open("PKGBUILD", encoding="utf-8").read() +assert "CMAKE_INSTALL_LIBDIR" not in s, "zstd: libdir already pinned, hook ran twice" +old = " -DCMAKE_INSTALL_PREFIX=/usr \\\n" +n = s.count(old) +assert n == 1, "zstd: expected 1 cmake configure, found %d" % n +s = s.replace(old, old + " -DCMAKE_INSTALL_LIBDIR=lib \\\n", 1) +io.open("PKGBUILD", "w", encoding="utf-8").write(s) +PY +[ "$(grep -c -- '-DCMAKE_INSTALL_LIBDIR=lib' PKGBUILD)" = 1 ] || { + echo "zstd: libdir not pinned exactly once" >&2; exit 1; } +echo "zstd: CMAKE_INSTALL_LIBDIR=lib (GNUInstallDirs would say multiarch)" diff --git a/scripts/bootstrap-pacman.sh b/scripts/bootstrap-pacman.sh index b359aa0..eea14f9 100755 --- a/scripts/bootstrap-pacman.sh +++ b/scripts/bootstrap-pacman.sh @@ -47,6 +47,21 @@ install_host_deps() { libgmp-dev libmpfr-dev libmpc-dev python3-docutils \ libseccomp-dev libpcre2-dev \ po4a gnat libdebuginfod-dev libjansson-dev + install_host_shims +} + +# The parts of "make the host look enough like Arch" that apt cannot express. +# +# Called from install_host_deps AND from build-stage1.sh, deliberately. The +# stand-ins are only consulted through /usr/local/bin, so editing the copy in +# this repository changes NOTHING until it is installed -- and a stale +# /usr/local copy is invisible: the build succeeds and ships the wrong paths. +# Re-installing them on every stage-1 run makes the repository the source of +# truth in fact, not just in intent. Both operations are idempotent. +install_host_shims() { + log "Host shims" + local d="$HERE_DIR/devtools" + sudo install -m755 "$d/arch-meson" "$d/arch-cmake" /usr/local/bin/ } build_pacman() { diff --git a/scripts/build-stage1.sh b/scripts/build-stage1.sh index f3056c7..0aa4e61 100755 --- a/scripts/build-stage1.sh +++ b/scripts/build-stage1.sh @@ -64,6 +64,10 @@ mark() { echo "$1" >> "$STATE"; } main() { mkdir -p "$WORK/pkg" "$REPO" touch "$STATE" + # The stand-ins are read from /usr/local/bin, so a stage-1 run that never + # reinstalls them silently builds with whatever was deployed weeks ago. + # Cheap, idempotent, and it makes this repository the source of truth. + install_host_shims local ok=0 fail=0 failed=() for p in "${STAGE1_PACKAGES[@]}"; do if built "$p"; then diff --git a/scripts/devtools/arch-meson b/scripts/devtools/arch-meson index 1de327d..1c01c00 100755 --- a/scripts/devtools/arch-meson +++ b/scripts/devtools/arch-meson @@ -9,8 +9,44 @@ # This reproduces the options devtools passes, which are what the PKGBUILDs # assume: /usr prefix, plain buildtype so makepkg's CFLAGS are honoured # verbatim, and no network access for subprojects. +# +# TWO OPTIONS BELOW ARE NOT IN REAL devtools, AND COPYING devtools VERBATIM +# IS THE BUG THEY FIX. On Arch these defaults are already right, so devtools +# has nothing to say. We run on Ubuntu, where meson answers differently. +# +# --libdir. meson's default (utils/universal.py, default_libdir): +# +# if is_debianlike(): # /etc/debian_version exists +# dpkg-architecture -qDEB_HOST_MULTIARCH +# return 'lib/' + archpath # -> lib/s390x-linux-gnu +# +# So every arch-meson package installed into /usr/lib/s390x-linux-gnu while +# its PKGBUILD went on naming /usr/lib. Nothing complained: makepkg exited 0 +# and the packages went into the repo with their libraries somewhere no Arch +# ld.so and no pkgconf will ever look. util-linux is where it finally shouted, +# +# rm: cannot remove '.../pkg/util-linux/usr/lib/lib*.a*': No such file +# +# and even there the rm was the first victim, not the cause -- the +# `mv usr/lib/lib*.so*` two lines below would have shipped an EMPTY +# util-linux-libs. Measured before the fix: expat, lz4, pacman, pkgconf and +# zstd were already in repo/s390x carrying multiarch paths, libalpm.so.16 +# among them. +# +# --libdir sits BEFORE "$@" so a PKGBUILD passing its own still wins: meson's +# argparse takes the last occurrence. +# +# python.platlibdir / python.purelibdir. The same divergence one layer up: +# meson's python module uses Debian's `deb_system` scheme and installs to +# /usr/lib/python3/dist-packages, which Arch's python does not read. The +# value is derived from the interpreter rather than hardcoded, because that +# is exactly how the PKGBUILDs derive theirs -- util-linux's package() moves +# `$(python -c 'sysconfig.get_paths()["stdlib"]')/site-packages`, so deriving +# it the same way makes the two agree by construction instead of by luck. +_pystdlib="$(python3 -c 'import sysconfig; print(sysconfig.get_paths()["stdlib"])')" exec meson setup \ --prefix /usr \ + --libdir lib \ --libexecdir lib \ --sbindir bin \ --buildtype plain \ @@ -18,4 +54,6 @@ exec meson setup \ --wrap-mode nodownload \ -D b_pie=true \ -D python.bytecompile=1 \ + -D python.platlibdir="$_pystdlib/site-packages" \ + -D python.purelibdir="$_pystdlib/site-packages" \ "$@"