From adc508d8f6c605a4c0038620b1637d284d8e3514 Mon Sep 17 00:00:00 2001 From: Mathieu Benoit Date: Wed, 19 Aug 2026 20:19:37 -0400 Subject: [PATCH] [FIX] python packages: /usr/local, once, for all of them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Debian patches sysconfig to prefer the "posix_local" scheme, so every wheel-installing PKGBUILD ships /usr/local -- a tree Arch reserves and whose every path the filesystem package owns. pacman refuses the transaction: error: failed to commit transaction (conflicting files) /usr/local/share/man exists in both 'filesystem' and 'meson' Four packages hit it. Two were dropped for other reasons. Rather than write a third and fourth relocation hook, build_package now exports DEB_PYTHON_INSTALL_LAYOUT=deb_system, which moves the default from /usr/local/lib/python3.13/dist-packages to /usr/lib/python3/dist-packages and scripts from /usr/local/bin to /usr/bin. One line, every python package, including ones not built yet. It does not finish the job: deb_system still says dist-packages and Arch reads site-packages. That move stays per-package, because only modules actually imported on the target need it. meson does, and its hook does it -- targeting the version of the python PACKAGE in repo/s390x, not the interpreter running the build. Installed under the host's 3.13, /usr/bin/meson runs in the chroot and then says ModuleNotFoundError, because the chroot's python is our 3.14 and never looks in 3.13. --- FR --- Debian modifie sysconfig pour préférer le schéma « posix_local » : tout PKGBUILD installant une roue livre donc /usr/local, arbre qu'Arch réserve et dont le paquet filesystem possède chaque chemin. pacman refuse la transaction : error: failed to commit transaction (conflicting files) /usr/local/share/man exists in both 'filesystem' et 'meson' Quatre paquets l'ont rencontré. Deux ont été écartés pour d'autres raisons. Plutôt que d'écrire un troisième et un quatrième crochet de relocalisation, build_package exporte désormais DEB_PYTHON_INSTALL_LAYOUT=deb_system, qui déplace le défaut de /usr/local/lib/python3.13/dist-packages vers /usr/lib/python3/dist-packages, et les scripts de /usr/local/bin vers /usr/bin. Une ligne, tous les paquets python, y compris ceux pas encore bâtis. Cela ne termine pas le travail : deb_system dit encore dist-packages quand Arch lit site-packages. Ce déplacement reste par paquet, car seuls les modules réellement importés sur la cible en ont besoin. meson en fait partie, et son crochet le fait — en visant la version du PAQUET python de repo/s390x, non l'interpréteur qui exécute la compilation. Installé sous le 3.13 de l'hôte, /usr/bin/meson démarre dans le chroot puis annonce ModuleNotFoundError, le python du chroot étant notre 3.14, qui ne regarde jamais dans 3.13. Assisted-by: Claude Opus 5 --- patches/pkgbuild/meson.sh | 91 +++++++++++++++++++++++++++++++++++++ scripts/bootstrap-pacman.sh | 27 +++++++++++ 2 files changed, 118 insertions(+) create mode 100755 patches/pkgbuild/meson.sh diff --git a/patches/pkgbuild/meson.sh b/patches/pkgbuild/meson.sh new file mode 100755 index 0000000..d97294d --- /dev/null +++ b/patches/pkgbuild/meson.sh @@ -0,0 +1,91 @@ +#!/usr/bin/env bash +# meson: `python -m installer` puts it under /usr/local. +# +# error: failed to commit transaction (conflicting files) +# /usr/local/share/man exists in both 'filesystem' and 'meson' +# +# Debian and Ubuntu patch sysconfig to prefer the "posix_local" scheme, so the +# host's python installs a wheel into /usr/local/lib/pythonX.Y/dist-packages +# with its scripts in /usr/local/bin. An Arch package may not ship /usr/local +# at all -- the filesystem package owns that tree, and ships +# usr/local/share/man as a SYMLINK, which is what the conflict is about: our +# meson shipped a directory where filesystem ships a link. +# +# THIRD INSTANCE of this scheme. python-brotli and python-libseccomp were +# simply dropped, because their wheels could not work on the target anyway. +# meson cannot be dropped -- stage 2 needs it to rebuild ten packages -- so it +# gets relocated instead. +# +# STAGE-1 ONLY, and listed in build-stage2.sh's STAGE2_SKIP_HOOKS. Inside the +# chroot the python doing the installing is OUR python, built from Arch's +# PKGBUILD, whose sysconfig answers /usr/lib/pythonX.Y/site-packages without +# being asked. The condition disappears, like git's missing zlib-ng headers and +# libgcrypt's too-old libgpg-error. +# +# AND THE VERSION IS THE TARGET'S, NOT THE HOST'S. This is the part that took +# a failed stage-2 build to see. meson is pure Python, so its ABI does not +# matter -- but the directory NAME carries a version, and the two pythons +# disagree: +# +# host python 3.13 -> /usr/lib/python3.13/site-packages +# our python pkg 3.14 -> /usr/lib/python3.14/site-packages +# +# Installed under the host's version, /usr/bin/meson runs in the chroot and +# then says +# +# ModuleNotFoundError: No module named 'mesonbuild' +# +# because the chroot's python -- ours -- never looks in 3.13. So the hook reads +# the version out of the python PACKAGE in repo/s390x, which is the python that +# will actually import this. Deriving it from the interpreter running the build +# is the intuitive thing and the wrong one. +set -euo pipefail +python3 - <<'ZZPY' +import io +s = io.open("PKGBUILD", encoding="utf-8").read() +old = ' python -m installer --destdir="$pkgdir" dist/*.whl\n' +assert s.count(old) == 1, "meson: installer invocation not in the expected form" +new = old + """ # Debian's posix_local scheme installs to /usr/local; Arch reserves that + # tree for the administrator and its filesystem package owns every path in + # it. Move what the wheel put there, then take the tree away. + local _repo="${REPO:-$HOME/work/arch-s390x/repo/s390x}" + # Two layouts to catch. build_package exports + # DEB_PYTHON_INSTALL_LAYOUT=deb_system, which keeps the wheel out of + # /usr/local -- but leaves it in dist-packages, which Arch's python does not + # read. The /usr/local branch stays for anyone running makepkg by hand + # without that variable. + # The TARGET python's version, read from our own package -- see the note at + # the top of this hook. Falls back to the host's only if the package is not + # built yet, which would be a stage-1 ordering error worth noticing. + local _pyver + _pyver=$(bsdtar -xOf "$_repo"/python-[0-9]*.pkg.tar.* .PKGINFO 2>/dev/null \ + | sed -n 's/^pkgver = \\([0-9]*\\.[0-9]*\\).*/\\1/p' | head -1) + if [ -z "$_pyver" ]; then + _pyver=$(python -c 'import sys; print("%d.%d" % sys.version_info[:2])') + echo " warning: no python package in $_repo, using host python $_pyver" >&2 + fi + local _site="$pkgdir/usr/lib/python$_pyver/site-packages" + if [ -d "$pkgdir/usr/local" ]; then + install -dm 755 "$pkgdir/usr/lib/python$_pyver" "$pkgdir/usr/bin" + [ -d "$pkgdir/usr/local/lib/python$_pyver/dist-packages" ] && \\ + mv "$pkgdir/usr/local/lib/python$_pyver/dist-packages" "$_site" + [ -d "$pkgdir/usr/local/bin" ] && mv "$pkgdir/usr/local/bin/"* "$pkgdir/usr/bin/" + rm -rf "$pkgdir/usr/local" + fi + if [ -d "$pkgdir/usr/lib/python3/dist-packages" ]; then + install -dm 755 "$pkgdir/usr/lib/python$_pyver" + if [ -d "$_site" ]; then + cp -a "$pkgdir/usr/lib/python3/dist-packages/." "$_site/" + rm -rf "$pkgdir/usr/lib/python3/dist-packages" + else + mv "$pkgdir/usr/lib/python3/dist-packages" "$_site" + fi + rmdir --ignore-fail-on-non-empty "$pkgdir/usr/lib/python3" 2>/dev/null || true + fi +""" +s = s.replace(old, new, 1) +io.open("PKGBUILD", "w", encoding="utf-8").write(s) +ZZPY +grep -q 'posix_local scheme installs' PKGBUILD || { + echo "meson: the relocation was not inserted" >&2; exit 1; } +echo "meson: wheel relocated out of /usr/local; STAGE-1 ONLY" diff --git a/scripts/bootstrap-pacman.sh b/scripts/bootstrap-pacman.sh index 4a27bbf..6377ee8 100755 --- a/scripts/bootstrap-pacman.sh +++ b/scripts/bootstrap-pacman.sh @@ -412,7 +412,34 @@ build_package() { # once, rather than in a hook per package. C.UTF-8 and not C: the Arabic # and Ukrainian pages must stay valid UTF-8. Arch's own build chroot runs # in this locale, so this makes us match it rather than diverge. + # DEB_PYTHON_INSTALL_LAYOUT=deb_system, because Debian's python installs + # into /usr/local and an Arch package may not ship that tree. + # + # $ python3 -c 'import sysconfig; print(sysconfig.get_default_scheme(), + # sysconfig.get_path("purelib"), sysconfig.get_path("scripts"))' + # posix_local /usr/local/lib/python3.13/dist-packages /usr/local/bin + # $ DEB_PYTHON_INSTALL_LAYOUT=deb_system python3 -c '...' + # deb_system /usr/lib/python3/dist-packages /usr/bin + # + # Without it every wheel-installing PKGBUILD ships /usr/local, and the + # filesystem package owns every path in there -- including + # usr/local/share/man, which it ships as a SYMLINK. pacman then refuses the + # whole transaction: + # + # error: failed to commit transaction (conflicting files) + # /usr/local/share/man exists in both 'filesystem' and 'meson' + # + # Four packages hit this before it was fixed here rather than in each of + # them: python-brotli and python-libseccomp (both dropped for other + # reasons), meson and python-tqdm. Setting it once covers every python + # package, including ones not built yet. + # + # It does NOT finish the job: deb_system still says dist-packages, and + # Arch's python reads site-packages. That last move stays per-package, + # because only the packages whose modules are actually IMPORTED on the + # target need it -- meson does, and its hook does it. LC_ALL=C.UTF-8 \ + DEB_PYTHON_INSTALL_LAYOUT=deb_system \ makepkg --nodeps --ignorearch --skippgpcheck --skipchecksums --nocheck \ -C -f || return 1 # An exit code is not proof. Only the artefact is.