Mathieu Benoit
469a5fde9e
Le plan cédait à l'enfant ce que le parent pouvait céder. Descente réelle à dix étages : l'étage 4 a reçu 44 Go et 2 vCPU sur un hôte qui en avait 2 — cent pour cent de surengagement, à chaque étage. Son installation dure 2 h 52 et n'est pas finie, contre 793 s pour l'étage 3. Extrapolé, le dixième demandait des années. Le plus profond reçoit désormais ce qu'un Proxmox de test demande, et chaque parent ajoute son seul surcoût : un vCPU, 2 Gio, 10 Go. Dix étages tiennent sur 11 vCPU et 22 Go au premier, contre 50 Go avant. Corrige aussi l'explication du gel à 12 vCPU : cette VM avait douze vCPU sur un hôte qui en avait deux. C'est le surengagement qui gèle, pas le douze. --- EN --- The plan handed the child whatever the parent could spare. Real ten-level descent: level 4 got 44 GB and 2 vCPU on a host that had 2 — a hundred percent overcommit, at every level. Its install has run 2h52 and is not done, against 793 s for level 3. Extrapolated, the tenth wanted years. The deepest level now gets what a test Proxmox asks for, and each parent adds its own overhead only: one vCPU, 2 GiB, 10 GB. Ten levels fit in 11 vCPU and 22 GB at the first, against 50 GB before. Also corrects the account of the 12-vCPU freeze: that VM had twelve vCPU on a host with two. Overcommit freezes, not the twelve. Assisted-by: claude-opus-5 (cherry picked from commit 7cda84bf391ee3ed36c5953ca4b86df44bd09e3b)
85 lines
4.1 KiB
Markdown
85 lines
4.1 KiB
Markdown
|
||
# LongTest — tests that create real machines
|
||
|
||
These are not unit tests. They create virtual machines, install systems on
|
||
them, and take hours. They live here and **not** in `test/`, which the unit
|
||
runner sweeps: `./script/test/run_unit_test.sh` must stay runnable in seconds
|
||
on any machine, including one without virtualisation.
|
||
|
||
Run them from the menu — `TODO › Execute › Test › Long tests` — or directly.
|
||
|
||
## deep_proxmox.py — how deep does Proxmox-in-Proxmox go?
|
||
|
||
The practicable nesting depth cannot be deduced, only measured — and one
|
||
measurement is not a measurement.
|
||
|
||
A manual look at one fourth-level VM found a guest **36 times slower than real
|
||
time** (583 seconds of wall clock for 16 seconds of guest time, each ACPI line
|
||
taking a second) and then a frozen kernel: identical RIP across three samples
|
||
two minutes apart, and **not one byte written** to disk.
|
||
|
||
Running this script **refuted the conclusion drawn from it**. Its own
|
||
fourth-level VM — 2 vCPU where the manual one had 12 — booted, installed, and
|
||
wrote gigabytes. What looked like a nesting ceiling was a *parallelism*
|
||
ceiling under nesting. That is exactly what the algorithm caps, and this is
|
||
how it stopped being a guess.
|
||
|
||
Which is the point of the script: a number obtained once, on one machine, in
|
||
one chain, is an anecdote.
|
||
|
||
```
|
||
./LongTest/deep_proxmox.py --depth 10 --dry-run # the plan, nothing created
|
||
./LongTest/deep_proxmox.py --depth 10 # hours
|
||
./LongTest/deep_proxmox.py --detruire # undo it
|
||
```
|
||
|
||
The descent is **uniform**. Every level, the first included, goes through the
|
||
same six steps: create, wait for ssh, install Proxmox, reboot and check the
|
||
kernel, bring pmxcfs back up, check the storage. Only creation differs —
|
||
libvirt locally, `qm` afterwards.
|
||
|
||
It sends **our** `install_proxmox.sh` over scp instead of letting the VM clone
|
||
the repository: it is our code we want to exercise, and the remote is often
|
||
behind the checkout — a fix absent from the remote made the same defect "come
|
||
back" on three VMs in a row.
|
||
|
||
### The resource algorithm — sized from the bottom up
|
||
|
||
The first version handed down whatever the parent could spare, and a real
|
||
descent showed what that costs. Level 4 ended up with 44 GB of memory and
|
||
2 vCPU **on a host that had 2** — a hundred percent overcommit, at every
|
||
level, with the hypervisor itself to serve on top. Its install ran past two
|
||
and a half hours against thirteen minutes for level 3, and extrapolating that
|
||
ratio gave five years for the tenth.
|
||
|
||
So the direction is reversed. The deepest level gets what a test Proxmox
|
||
actually asks for — 4 GB of memory, 25 GB of disk, 2 vCPU — and every parent
|
||
above it adds its own overhead and nothing else: one vCPU, 2 GiB, 10 GB. A
|
||
ten-level descent therefore asks its first level for 11 vCPU, 22 GB and
|
||
115 GB, where handing resources down wanted 50 GB of memory for the same
|
||
depth.
|
||
|
||
Three budgets can bound the depth, and `script/proxmox/nesting.py` names the
|
||
one that ran out:
|
||
|
||
* **memory** — every level must run its own daemons (`pve-cluster`,
|
||
`pvestatd`, `pvedaemon`, `pveproxy`) *and* hold its child;
|
||
* **disk** — the child's disk lives *inside* the parent's, which must also
|
||
hold its own system;
|
||
* **processor** — each level wants one vCPU more than its child, so ten levels
|
||
ask eleven of the first. Half the physical cores is the ceiling: the
|
||
orchestrator runs on that machine too.
|
||
|
||
That third budget is measured, not assumed. Twelve vCPU at the fourth level
|
||
froze the guest kernel in early boot — same instruction pointer at three
|
||
readings two minutes apart — while two progressed. The number was not the
|
||
culprit: that VM had twelve vCPU on a host with two, six times wider than its
|
||
own machine. Overcommit freezes, not the twelve.
|
||
|
||
Memory is not the lever. On that same manual VM, dropping it from 9 GB to 2 GB
|
||
moved nothing — it stopped after reading the same 32 MiB, which is simply the
|
||
size of the boot files.
|
||
|
||
The plan is printed **before** anything is created, and the script never
|
||
promises a depth it knows will not fit — better to announce six levels and
|
||
reach six than to promise ten and die at the seventh without knowing why.
|