[FIX] stage 2: the build tools the host was providing silently

glibc rebuilt inside the chroot; binutils died on `make info-recursive`, and
makeinfo said why: its XS module was compiled against the host's perl 5.40 and
our perl is 5.42. Stage-1 texinfo cannot run in there. Everything that builds
.info documentation needs it, and it sat near the end of the list because that
is where its own dependencies are, so stage 2 hoists it.

linux-api-headers stopped earlier on `rsync: command not found` -- the kernel's
headers_install runs it. apt had put it there and a build that finds its tool
says nothing, so stage 1 never mentioned it. Its man pages come from Markdown
the git tag does not pre-render, hence --disable-md2man.

An inventory of the 110 apt packages against the chroot found 84 more such
binaries. Most are documentation; the rest wait until something needs them.

--- FR ---

glibc a été rebâti dans le chroot ; binutils est mort sur `make info-recursive`,
et makeinfo en donne la raison : son module XS a été compilé contre le perl 5.40
de l'hôte, or le nôtre est en 5.42. Le texinfo de l'étage 1 ne peut pas tourner
là-dedans. Tout ce qui bâtit de la documentation .info en dépend, et il figurait
en fin de liste, là où sont ses propres dépendances : l'étage 2 le remonte.

linux-api-headers s'était arrêté avant sur `rsync: command not found` — le
headers_install du noyau l'appelle. apt l'avait posé, et une construction qui
trouve son outil n'en dit rien : l'étage 1 ne l'a jamais mentionné. Ses pages de
manuel viennent d'un Markdown que l'étiquette git ne rend pas, d'où
--disable-md2man.

Un inventaire des 110 paquets apt face au chroot a trouvé 84 autres binaires du
même genre. La plupart sont de la documentation ; le reste attendra qu'un paquet
les réclame.

Assisted-by: Claude Opus 5

patches/pkgbuild/rsync.sh

@@ -0,0 +1,60 @@
+#!/usr/bin/env bash
+# rsync: no man pages, because nothing here can render them.
+#
+#   - You need python3 and either the cmarkgfm OR commonmark python3 lib in
+#     order to build manpages based on the git source
+#   configure.sh: error: Aborting configure run
+#
+# rsync is built from the git tag, and the git tree ships man pages as Markdown
+# rather than nroff -- the release tarballs carry them pre-rendered, the tag
+# does not. Rendering wants a Python Markdown library that this port does not
+# have at either stage: not on the host, and not in the chroot, where there is
+# no pip to reach for.
+#
+# --disable-md2man is named by the error itself. Same arbitration as
+# --auto-features auto in build-stage2.sh: the documentation gives way and the
+# tool gets built. Restore at stage 3, alongside the rest of it.
+#
+# WHY rsync IS HERE AT ALL. It is not part of Arch's base and nothing in this
+# port depends on it at runtime. It is a BUILD tool: the kernel's
+# headers_install target runs it, so linux-api-headers -- the first entry in
+# packages.sh -- cannot be rebuilt inside the chroot without it. The host had
+# it from apt, which is precisely why stage 1 never mentioned it.
+#
+# BOTH STAGES, no skip entry: the chroot has no Markdown renderer either, and
+# stage 2 is where rsync is actually needed.
+set -euo pipefail
+python3 - <<'ZZPY'
+import io, re
+# LINE-BASED, and [ \t]* rather than \s*.
+#
+# The first attempt matched r"^(\s*)\./configure" and inserted at the match
+# end. It produced
+#
+#   ./configure \
+#                        <- blank line
+#       --disable-md2man \
+#
+# and the guard below caught it. \s MATCHES NEWLINES: with re.M the pattern
+# could start at the blank line above, swallow "\n  " into the captured
+# indent, and put that newline back into the file. The flag was in the file,
+# on its own paragraph, attached to nothing.
+#
+# Working on the LINE removes the arithmetic entirely -- there is no offset to
+# be wrong about.
+lines = io.open("PKGBUILD", encoding="utf-8").read().split("\n")
+hit = [i for i, l in enumerate(lines) if re.match(r"^[ \t]*\./configure\b", l)]
+assert len(hit) == 1, "rsync: expected exactly one ./configure line, got %d" % len(hit)
+i = hit[0]
+indent = re.match(r"^[ \t]*", lines[i]).group(0)
+assert lines[i].rstrip().endswith("\\"), "rsync: ./configure does not continue onto the next line"
+lines.insert(i + 1, indent + "    --disable-md2man \\")
+io.open("PKGBUILD", "w", encoding="utf-8").write("\n".join(lines))
+ZZPY
+grep -q -- '--disable-md2man' PKGBUILD || {
+    echo "rsync: --disable-md2man was not inserted" >&2; exit 1; }
+# The insert must have landed on the configure line, not merely somewhere in the
+# file -- a flag in a comment or a later block would look identical to grep.
+grep -A1 -E '^\s*\./configure' PKGBUILD | grep -q -- '--disable-md2man' || {
+    echo "rsync: the flag is in the file but not on ./configure" >&2; exit 1; }
+echo "rsync: man pages disabled (no Markdown renderer at either stage)"

scripts/bootstrap-pacman.sh

@@ -475,48 +475,3 @@ main() {
 if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
     main "$@"
 fi
-
-# Run a command, and kill it if it stops saying anything.
-#
-#   watched <logfile> <command...>
-#
-# Returns the command's status, or 2 if it was killed for silence.
-#
-# WHY IT EXISTS. python's build ran for ten hours. It was not slow: it was
-# spinning in a shell loop waiting for an X server that this port will never
-# have, printing nothing. Nothing distinguished it from a long compile except
-# that the log had not grown -- and that is a measurement, so it can be made
-# automatically.
-#
-# Both stages need it, and stage 2 needs it MORE: stage 2 runs the test suites
-# that stage 1 skipped, and a test suite is the likeliest thing in a build to
-# wait forever on a terminal, a network socket, or a display.
-#
-# Killed as a PROCESS GROUP -- set -m gives the subshell its own, so make and
-# every compiler under it die too. Killing the pid alone leaves the tree
-# running, holding the disk, invisible to the driver that thinks it stopped it.
-watched() {
-    local log="$1"; shift
-    local limit="${EL_STALL_MIN:-45}"
-    if [ "$limit" -eq 0 ]; then
-        "$@" > "$log" 2>&1
-        return $?
-    fi
-    set -m
-    ( "$@" ) > "$log" 2>&1 &
-    local pid=$! last=0 still=0 sz
-    set +m
-    while kill -0 "$pid" 2>/dev/null; do
-        sleep 60
-        sz=$(stat -c %s "$log" 2>/dev/null || echo 0)
-        if [ "$sz" -eq "$last" ]; then still=$((still + 1)); else still=0; fi
-        last="$sz"
-        if [ "$still" -ge "$limit" ]; then
-            printf '\n== driver: no output for %s minutes, killing ==\n' "$limit" >> "$log"
-            kill -9 -- "-$pid" 2>/dev/null
-            wait "$pid" 2>/dev/null
-            return 2
-        fi
-    done
-    wait "$pid"
-}

scripts/build-stage1.sh

@@ -18,6 +18,7 @@ set -uo pipefail
 
 HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 source "$HERE/bootstrap-pacman.sh"
+source "$HERE/lib-watch.sh"
 
 WORK="${WORK:-$HOME/work/arch-s390x}"
 REPO="${REPO:-$WORK/repo/s390x}"

scripts/build-stage2.sh

@@ -30,6 +30,8 @@ HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 # Stage 2 rebuilds the same packages in the same order; packages.sh explains
 # why that order is not re-derived here.
 source "$HERE/packages.sh"
+# watched(): bootstrap-pacman.sh cannot be sourced here -- see lib-watch.sh.
+source "$HERE/lib-watch.sh"
 WORK="${WORK:-$HOME/work/arch-s390x}"
 REPO1="${REPO1:-$WORK/repo/s390x}"
 REPO2="${REPO2:-$WORK/repo2/s390x}"
@@ -55,6 +57,11 @@ CHROOT_PKGS=(
     bison flex texinfo groff gettext
     # makepkg itself, and the one thing it cannot do without.
     pacman fakeroot
+    # rsync, which is not a runtime dependency of anything here. The kernel's
+    # headers_install target runs it, so linux-api-headers -- the first package
+    # stage 2 tries -- stopped on `rsync: command not found`. The host had it
+    # from apt, which is exactly why stage 1 never mentioned it.
+    rsync
     # libxcrypt-compat, for a reason no declaration expresses. perl declares
     # `libxcrypt` and `libcrypt.so`, both satisfied by libxcrypt, which ships
     # libcrypt.so.2. But perl's BINARY was linked on the host against Ubuntu's
@@ -381,6 +388,31 @@ NOTE
 # host's Debian-patched python puts there.
 # libarchive: stage 1 drops xar to break the bsdtar -> libxml2 -> libicuuc.so.76
 # cycle. In here the versions agree, so build it as Arch does.
+# Hoisted to the front of the stage-2 order.
+#
+# Stage 1's order is the real dependency closure and stage 2 keeps it -- but
+# stage 1 had the HOST for its build tools, and a few of those tools are
+# themselves in the list, sitting wherever their runtime dependencies put them.
+# In the chroot the stage-1 copy is what is available, and for these it does
+# not work:
+#
+#   texinfo -- binutils died on `make info-recursive`, and makeinfo says why:
+#
+#     Perl API version v5.40.0 of Texinfo::TreeElement does not match v5.42.0
+#
+#   Its XS module was compiled against the HOST's perl 5.40; our perl package
+#   is 5.42. Every package that builds .info documentation -- binutils, gcc,
+#   glibc, coreutils, gettext, m4 -- needs makeinfo, and texinfo sits near the
+#   end of the list because that is where its own dependencies are. Rebuilding
+#   it in here against our perl fixes it, which is the entire point of stage 2;
+#   it just has to happen first.
+#
+# Duplicates need no handling: these names appear again later in
+# STAGE1_PACKAGES, and rebuild() skips anything already in stage2.state. The
+# list is prepended rather than reordered, so stage 1's order stays the single
+# statement of the closure.
+STAGE2_FIRST=(texinfo)
+
 STAGE2_SKIP_HOOKS=(libgcrypt git meson libarchive)
 
 # Packages whose sources must be extracted on the HOST, because the chroot
@@ -593,7 +625,7 @@ main() {
     # names is not a workflow, and typing a subset of them is how an ordering
     # gets quietly reinvented.
     if [ "$1" = "--all" ]; then
-        rebuild "${STAGE1_PACKAGES[@]}"
+        rebuild "${STAGE2_FIRST[@]}" "${STAGE1_PACKAGES[@]}"
     else
         rebuild "$@"
     fi

scripts/lib-watch.sh

@@ -0,0 +1,57 @@
+#!/usr/bin/env bash
+# The stall guard, and nothing else.
+#
+# WHY ITS OWN FILE. It belongs to both stages, so the obvious home was
+# bootstrap-pacman.sh, which stage 1 already sources. Stage 2 cannot source
+# that file: it opens with `set -euo pipefail`, and stage 2's whole rebuild
+# loop depends on a failed package NOT ending the run -- one missing package
+# must not hide the state of the hundred that follow. It also defines a main(),
+# which is a second thing to be careful about for no benefit.
+#
+# So: one file, no `set -e`, no side effects, sourced by both. Nothing else
+# goes in here.
+
+# Run a command, and kill it if it stops saying anything.
+#
+#   watched <logfile> <command...>
+#
+# Returns the command's status, or 2 if it was killed for silence.
+#
+# WHY IT EXISTS. python's build ran for ten hours. It was not slow: it was
+# spinning in a shell loop waiting for an X server that this port will never
+# have, printing nothing. Nothing distinguished it from a long compile except
+# that the log had not grown -- and that is a measurement, so it can be made
+# automatically.
+#
+# Both stages need it, and stage 2 needs it MORE: stage 2 runs the test suites
+# that stage 1 skipped, and a test suite is the likeliest thing in a build to
+# wait forever on a terminal, a network socket, or a display.
+#
+# Killed as a PROCESS GROUP -- set -m gives the subshell its own, so make and
+# every compiler under it die too. Killing the pid alone leaves the tree
+# running, holding the disk, invisible to the driver that thinks it stopped it.
+watched() {
+    local log="$1"; shift
+    local limit="${EL_STALL_MIN:-45}"
+    if [ "$limit" -eq 0 ]; then
+        "$@" > "$log" 2>&1
+        return $?
+    fi
+    set -m
+    ( "$@" ) > "$log" 2>&1 &
+    local pid=$! last=0 still=0 sz
+    set +m
+    while kill -0 "$pid" 2>/dev/null; do
+        sleep 60
+        sz=$(stat -c %s "$log" 2>/dev/null || echo 0)
+        if [ "$sz" -eq "$last" ]; then still=$((still + 1)); else still=0; fi
+        last="$sz"
+        if [ "$still" -ge "$limit" ]; then
+            printf '\n== driver: no output for %s minutes, killing ==\n' "$limit" >> "$log"
+            kill -9 -- "-$pid" 2>/dev/null
+            wait "$pid" 2>/dev/null
+            return 2
+        fi
+    done
+    wait "$pid"
+}

scripts/packages.sh

@@ -187,4 +187,19 @@ STAGE1_PACKAGES=(
     cppdap jsoncpp libuv rhash hicolor-icon-theme cmake
     # And finally the package manager itself, built as an Arch package.
     pacman
+
+    # Tools the STAGE-2 CHROOT needs, which the host supplied for free.
+    #
+    # Stage 1 never noticed these: install_host_deps put them there with apt,
+    # and a build that finds its tool does not mention it. Inside the chroot
+    # there is no apt and no host, so every one of them has to exist as a
+    # package -- and the first thing stage 2 tried to rebuild stopped on
+    #
+    #   /bin/sh: line 1: rsync: command not found
+    #   make[1]: *** [.../Makefile:1463: headers_install] Error 127
+    #
+    # from linux-api-headers, the first entry in this list. xxhash is here
+    # because rsync links it.
+    xxhash
+    rsync
 )