Writing ~/.ssh/config only helps if the key is accepted at the other end.
Both SSH options now offer to create an ed25519 key when none exists — the
same choice deploy_qemu.ensure_ssh_key makes, so created and adopted VMs share
one key — and to push it with ssh-copy-id.
Hosts that already accept the key are skipped, tested with
PasswordAuthentication=no: without it ssh would fall back to the password and
every host would look like it already had the key. ssh-copy-id runs on the
real terminal rather than through captured output, otherwise its password
prompt would be invisible.
In the recursive walk the key is deployed BEFORE probing each level, not at
the end. The probe uses BatchMode, so an un-keyed machine answers nothing and
the level below it stays invisible — deploying afterwards would find only the
first level.
virt-manager, when installed, gets the machines that actually run libvirt
added to its connection list, so their nested VMs can be driven from the local
GUI. The URI uses the SSH ALIAS rather than the raw IP: qemu+ssh goes through
the ssh binary, hence ~/.ssh/config, so the alias already carries both the
address and the ProxyJump — a bare IP could not reach a nested VM at all.
Connections live in GSettings, not a file. The list is READ first and written
back merged, so nothing already configured is lost, and a failed read (no
schema, no virt-manager) simply means the whole feature stays silent — no
prompt, no write. virt-manager rewrites its settings when it exits, so a
warning says to restart it.
Verified: key generated with 0600 on the private half and reused on the second
call; ssh-copy-id issued only for hosts that need it; the recursive walk
deploying level by level before each probe; GSettings merge keeping existing
URIs and skipping the write when nothing is missing; « @as [] », a populated
list and a missing schema all parsed correctly.
Caveat: virt-manager is not installed on this machine, so the absent path was
exercised for real and the write path only against a stubbed gsettings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
~/.ssh/config entries were only ever written while deploying a VM. A fleet
that already exists — or one whose DHCP leases have moved — had no way to
refresh them. Entry [13] of the QEMU menu does it on demand:
[1] update ~/.ssh/config for the local VMs
[2] add the nested VMs through ProxyJump, recursively
The second one matters because of the « ERPLibre Deployment (+ QEMU + dev) »
profile: a VM built that way hosts VMs of its own, on its own private network.
Those are not reachable from this host at all — only from their parent. So
each level is written with a ProxyJump to the level above, and OpenSSH chains
the hops on its own. « ssh erplibre-fedora-42 » then works from here even
though the address only means something two machines away.
The recursion probes over « ssh <alias> », i.e. through the block just
written, so the parent's own ProxyJump applies automatically and one probe
works identically at any depth. One SSH connection per MACHINE, not per VM: a
single snippet returns every « name<TAB>ip » pair, falling back to the guest
agent when the dnsmasq lease is missing. Passwordless sudo is a given here —
the cloud-init config grants it (deploy_qemu.py:1175).
A nested VM keeps its short name, which is what one wants to type, and is only
prefixed with its parent on collision — so discovering a machine that already
exists elsewhere never overwrites the other one's entry. Depth defaults to 2
(host, VM, nested VM) and already-seen aliases are skipped, which is what
stops a cycle: a child that reports its own grandparent.
Verified on a simulated two-level fleet including a deliberate cycle: the
ProxyJump chain is correct at each level, the colliding name is prefixed, the
parent block is not overwritten, and only one probe per machine is issued. The
remote snippet itself was run for real on this host — valid POSIX sh, two VMs
with their addresses. A VM without an IP is skipped rather than written with
an empty HostName.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Uninstalling before the 13->14 bump asked for 15 modules; 12 of them no longer
exist in the 13.0 addons path. check_addons_exist.py rejects the command on
the first missing name, so the uninstall aborted whole — and the 3 modules
that WERE there stayed installed, blocking the bump for a reason that had
nothing to do with them.
The list is now split before the script is called, using check_addons_exist
which was already there and simply never consulted at this point. « Missing »
means the addons path has no code for it, not that the database lacks it —
the distinction is the whole point, since Odoo cannot uninstall a module whose
code is gone.
When some are missing, a choice is offered rather than a failure:
[1] uninstall the present ones, skip the missing (default)
[2] try the whole list anyway (it will fail)
[3] uninstall nothing, continue
Only what is really uninstalled is subtracted from the per-version module
list, so a module left in place stays counted as installed — which it is. The
missing names are written to the progression comments, so what was skipped is
still on record afterwards.
Verified on the exact list from the failure: 3 present and 12 missing,
option 1 issuing a command with only the 3 and leaving the 12 counted as
installed, option 2 sending all 15, option 3 running nothing.
Also swapped entries 4 and 5 of the Deploy menu — QEMU/KVM now sits at [4],
NTFY at [5].
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The migration now opens with the same choice the QEMU deployment does — TUI
or line-by-line prompts — settled in advance by TODO > Configuration if you
want it to be. Choosing the TUI loads the saved progression and asks the very
same first question: where does this migration stand, and where do we resume?
prompt_resume was one function that rendered, read and decided at once, so a
second view would have meant a second copy of the decision. It is now three:
resume_context() the progression -> plain data (file, database, target,
steps with their icon and detail, version bumps)
print_resume() renders it on the terminal
apply_resume_answer() answer -> (progression, changed)
The TUI returns the SAME answer strings as the prompt — c, n, r, q, 0..4,
4.<version> — so apply_resume_answer stays the only place that decides what a
choice means. Neither view can drift into describing the migration
differently, because both render the same context.
In the TUI the steps are a table and the version bumps a list: Enter on either
replays from there, which is what « [0-4] » and « [4.N] » meant in text. The
cursor opens on the first unfinished step and on the first unmigrated version
— where it stopped is where one usually wants to act.
Both views gain « q », quit without doing anything. A TUI needs Escape to do
something sane, and an escape hatch present in only one of the two views is
exactly the kind of divergence this split exists to prevent. execute_odoo_
upgrade returns immediately on it, writing nothing.
Verified on a 12->18 progression with steps 0-3 done and 2 of 6 bumps
migrated: identical context feeding both views, Enter on step 2 giving « 2 »,
Enter on the 15 bump giving « 4.15 », the c/n/r/q/Escape shortcuts, and every
answer producing the same progression through both paths — « 2 » leaving only
the state of steps 0 and 1, « 4.15 » resetting the clone list from the third
bump on so the half-migrated intermediate database gets rebuilt. « q » checked
to leave the progression file byte-identical.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The form opened with the four main versions already ticked, so the plan showed
four VMs nobody had asked for. F5 on an untouched form would have created
them. Deploying is expensive and hard to undo — the list now starts empty and
the choice is made, not inherited.
The « * » still marks each distro's main version, and F7 ticks exactly those
four in one keystroke, so nothing is lost but the presumption.
With an empty list a « 0 VM · 0 vCPU » total teaches nothing, so the footer
says what to do instead: tick what to deploy, F7 main versions, F6 all.
Verified headless: no box ticked on open, F5 producing no spec at all, F7
giving back the four, and the ticks surviving a switch to « all
architectures » (4 of 30) since identity is distro/version/arch, not rank.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deploying a VM meant answering a dozen questions in a row, never seeing the
whole of one's choices, and starting over to revisit an answer. The first
question is now which interface to use — TUI form or the classic prompts —
defaulting to whatever TODO > Configuration says.
The form shows every setting at once with a plan that recomputes on each
change: names, resources, and the two collisions marked as they arise (a
defined VM is skipped, an orphan qcow2 will make deploy_qemu fail). F2 edits
one VM, F3 previews the commands, F5 deploys, F6/F7/F8 select all / main
versions / none.
Function keys rather than ctrl+letter: ctrl+p is Textual's command palette and
silently swallowed the shortcut, and a bare letter is eaten by whichever input
has focus.
Both interfaces go through _qemu_deploy_parts_for, so the same choices produce
the same command by construction — and a test now drives the CLI prompts and
the form to the same state and compares the argv, which is what keeps them
from drifting.
Nothing privileged or networked runs inside the form. Every virsh call in this
codebase goes through sudo, and a password prompt while Textual owns the
terminal would wreck the display; the domain list and the remote branches are
fetched before the app starts and arrive as plain data.
The progress view is optional (preference: CLI output stays the default, since
it is the easiest to copy from). It gives one collapsible block per VM,
expanded while running, folded on success — and left OPEN on failure, which is
the part worth reading. « c » copies the selected log, « C » all of them.
On copying from a TUI over SSH: copy_to_clipboard emits OSC 52, which the
LOCAL terminal emulator interprets, so it does reach the workstation's
clipboard. Two caveats are handled: the payload is capped at 100 kB keeping
the TAIL (some terminals truncate long OSC 52), and the notification says a
compatible terminal is required — macOS Terminal.app has none, tmux needs
set-clipboard on.
Verified: pure logic (profiles, per-VM overrides surviving a profile change,
totals, statuses) by direct calls; the form headless via run_test — default
selection, F6/F7/F8, profile switch, and the orphan guard demanding a second
F5; parity CLI/TUI on identical argv; the progress view on three fake jobs,
with the failure staying open and the clipboard filled. The semaphore bounding
concurrency was measured: four 0.35 s jobs take 1.63 s at 1 and 0.58 s at 4 —
without it, « 4 in parallel » launched every VM at once.
Also removed a duplicated @staticmethod on _qemu_install_dir, harmless since
Python 3.10 but misleading.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_qemu_deploy was ~400 lines mixing a dozen prompts with the parallel
deployment, IP resolution, ~/.ssh/config and the ERPLibre install. A second
interface could not be added to it without duplicating all of that — and the
repository already shows where duplication leads: _qemu_choose_cli_browser
(todo.py) and Monitor._choose_browser (qemu_install_monitor.py) are the same
function twice, and they have already drifted apart.
The function now has three parts around a plain dict, the SPEC:
_qemu_collect_vms_cli arch, catalog, resources, names -> the VM list
_qemu_collect_options_cli SSH key, install, parallelism -> the spec
_qemu_run_spec consumes a spec, asks nothing
_qemu_deploy_parts_for is the single point every command goes through, so two
interfaces producing the same spec necessarily produce the same command —
which makes their divergence testable rather than a matter of discipline.
Pure, I/O-free helpers come out of the body: _qemu_catalog_entries (the flat
distro × version × arch list), _qemu_arches_for, _qemu_make_vm,
_qemu_split_existing and _qemu_orphan_disks. The last two matter beyond
tidiness — the form must recompute collisions on every keystroke, and every
virsh call in this file goes through sudo. Existence is now resolved with ONE
virsh list --all --name (_qemu_list_domains, already present) instead of one
sudo per VM, and the orphan-disk scan needs no privileges at all.
VMs travel as dicts rather than 6-tuples plus a parallel names list. The
tuple-based resource prompts are left untouched and converted at the boundary.
No behaviour change intended, and verified as such: replaying identical
scripted answers against a worktree pinned at the previous commit produces
byte-identical output — including the granular selection spanning three
architectures — once the repository path and the instantaneous free-RAM
reading are normalised.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Nothing in the CLI could be remembered from one session to the next except the
language, which lives in env_var.sh with its own ad-hoc parser. Entry [6]
Configuration, below Telemetry, now groups the settings that belong to the
USER rather than to the repository.
todo_prefs.py stores them in ~/.erplibre/todo_prefs.json — the same place and
the same best-effort shape as todo_telemetry.py, since both are per-user and
per-machine, and neither must ever prevent the CLI from starting. A DEFAULTS
table gives every known key its fallback, so a missing or corrupt file simply
reads as defaults.
The two keys added here prepare the QEMU deploy form: which interface to use
(ask / TUI / classic) and what to display while deploying (CLI output or TUI).
They are declared once in _PREF_CHOICES — the screen, the current-value label
and the editor all derive from that single table, so adding a preference is
one entry, not three edits.
Language keeps its own mechanism: it is read before the preferences file
exists and is consumed by shell scripts too.
Verified against a temporary HOME: defaults returned with no file on disk, a
change persisted and reflected in the menu, and reset falling back to
defaults.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resources per VM offered x1..x4 only — anything else meant deploying with the
wrong sizing and resizing afterwards. « [5] Custom » now asks vCPU, RAM and
disk once for the whole fleet, and vCPU joins disk and RAM in the per-VM
customisation. Presets 1, 2, 4, 6, 8, 16, 24, 32.
The two places ask the SAME three questions, so they are one definition each
(_qemu_ask_disk / _qemu_ask_ram / _qemu_ask_cpu) instead of two copies that
would drift. vCPU is no longer a single global value: it travels per VM
through the whole chain, down to --vcpus.
Overcommit is allowed on an explicit choice — KVM permits more vCPU than
cores — and only warned about. The x1..x4 path still caps at the host core
count: that one is an automatic computation, not a decision.
Three prompts default to yes (install ERPLibre, ~/.ssh/config, deploy): they
are what one wants nearly every time, and typing « o » on each was noise.
Flipping a destructive-by-omission default demanded the guards below.
Before deploying, a final review lists everything that will change — VMs to
create with their effective disk (ERPLibre's +5 G included), VMs left
untouched, install profile and branch, SSH key, ~/.ssh/config, parallelism.
Answering no asks for confirmation rather than dropping every answer given
over a dozen prompts.
Name collisions are now reported BEFORE the wait, with their two very
different consequences: an already-defined VM is skipped and nothing is
overwritten, while a qcow2 left behind by a deleted VM makes deploy_qemu fail,
since it refuses to overwrite without --force. Continuing requires an explicit
yes; the default is no.
Also translated two strings of this flow that had no entry and stayed in
English (« Resolving VM IPs… », « no IP »).
Verified by driving the prompts with scripted answers: custom profile applied
and left intact when blank, per-VM override, 32 vCPU on 28 cores warned but
accepted, review with and without ERPLibre install, no → no → deploy, no →
yes → cancel, and collisions defaulting to no. Rendering checked in fr and en.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The install history was only surfaced as a « ~5m avg (3) » suffix next to a
distro. Entry [12] now shows what that history actually contains: totals and
success rate, period covered, median/min/max and cumulated duration, then a
breakdown by distribution, version and architecture, plus the current VMs and
the disk they occupy. « [r] » erases the history after confirmation.
record_duration() was only called on success, so no success rate could ever be
computed. Failures are now recorded with ok=False. They are counted separately
and EXCLUDED from the averages and the ETA: how long a failed install ran says
nothing about how long a successful one takes. Entries written before the flag
existed have no « ok » key and are read as successes, which they were.
Aggregation lives in qemu_install_monitor.py as pure functions (stats_summary,
stats_by, all_runs, reset_stats), display in todo.py — the split the file
already follows.
Disk presets extended to 400G, 600G, 800G, 1T, 1.5T and 2T. The parser only
understood G, so « 1T » would have been rejected: sizes are now normalised
through _qemu_parse_disk (1 T = 1024 G, decimal comma accepted), since the rest
of the chain reasons in gigabytes.
Verified with a synthetic history of 9 runs including 2 failures: the rate,
the per-group failure counts and the reset all behave; a group with no success
shows « — » rather than a misleading « ~0s ».
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The presets stopped at 32768 MB; current virtualization hosts go well beyond
that. The series now doubles up to 262144 MB (256G).
The prompt is in MB while people think in GB, so each preset carries its
equivalent: « [g] 65536 (64G) ». With nine of them the line no longer fits, so
suggestions are laid out five per row. Disk presets keep the plain format.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both prompts expected a raw number, so common values had to be retyped every
time. They now list suggestions:
[a] 20G [b] 40G [c] 60G [d] 80G [e] 120G [f] 200G
New disk size in G, blank = keep (20G):
Letters start at « a » so they can never collide with a value typed directly:
anything starting with a digit is read as the value itself, and blank still
keeps the current one. A letter outside the range is rejected instead of being
silently taken for a size.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_qemu_default_ssh_key() returns '' when the user has no public key, so the
deployment injects none through cloud-init: the VM boots with no SSH access and
its state can no longer be checked once created.
--setup-host now generates an ed25519 key without passphrase when neither
id_ed25519.pub nor id_rsa.pub is present, so the Deployment profile leaves a
host able to reach the VMs it creates.
The key is created AS the invoking user via « sudo -u », not as root: under
sudo « ~ » is /root and the key would land where cloud-init never looks.
Verified on a fresh Arch VM with no key: it is created as erplibre:erplibre
with 700/600/644, a rerun keeps the same fingerprint, and
_qemu_default_ssh_key() now returns it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When OpenUpgrade failed, todo_upgrade_execute offered the generic
« [1] to redo the command ». Replaying it re-ran OpenUpgrade on the database it
had just half migrated, which never recovers. That call now passes
wait_at_error=False, and the failure message explains that the clone step has
been reset so a relaunch DROPS and REBUILDS the intermediate database from the
previous version.
The resume menu gains « [4.N] »: replay the step 4 loop from one version bump.
Only the per-version lists are trimmed from that index, so earlier bumps stay
migrated and steps 0 to 3 are untouched. Resetting the clone entry is the whole
point: the intermediate database of a failed bump must be rebuilt, not upgraded
again. « [4] » still replays every bump.
The per-bump uninstall files were also read at step 1 only, for the source
version, so uninstall_module_list_odoo130_to_odoo140.txt existed in name but
was never read. The step 4 loop now reads the file of the bump it is about to
perform, merged with the answers already stored in the progression.
Verified against the real progression (13 and 14 migrated): the menu offers
13/14/15/16/17/18, and replaying from 14 turns every per-version list from
[True, True, False...] into [True, False, ...] while state_4_reach_open_upgrade
and steps 0-3 survive. A 13->14 list of 4 modules is read with its reasons.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 13.0 -> 14.0 data migration died on:
duplicate key value violates unique constraint "res_groups_name_uniq"
Key (category_id, name)=(9, Allow to define fiscal years of more or less
than a year) already exists
In 13.0 that security group is declared by the core « account » module, so the
database holds it as account.group_fiscal_year. In 14.0 core account no longer
declares it and om_account_accountant (odoomates) does. Owning no XML id of its
own, the module tries to CREATE the group and collides with the existing row.
Renaming the XML id makes Odoo UPDATE the existing record instead. The record
id is untouched, so user assignments, access rights and record rules pointing
at the group survive -- on the reference database it holds none, but the fix
must not depend on that.
The fix hook also learns a « .sql » flavour, run through psql. The « .py »
flavour is piped into « odoo<target>-bin shell », which requires loading a
not-yet-migrated database with the TARGET version's registry -- precisely what
is failing at that point. A pure SQL fix needs no ORM and no registry.
Verified end to end on a throwaway copy of the stuck database: the statement
renames one row, a second run changes nothing, and the full 13->14 OpenUpgrade
then completes (« Modules loaded. », base at 14.0.1.3, zero
res_groups_name_uniq error).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On a rolling distro every fresh development VM was born unable to create VMs.
« make install_os » runs a full system upgrade, the kernel package is replaced
and /lib/modules/<running kernel> disappears. modprobe bridge then fails and
libvirt cannot create virbr0, so the « default » network stays inactive and
virt-install dies on « network 'default' is not active » -- with every package
correctly installed. Measured twice on a freshly created Arch VM: booted on
7.1.3-arch1-3 at 05:44, upgraded to 7.1.5.arch1-2 at 05:46.
Only a reboot fixes it, and one is enough: the default network is already
flagged autostart, so libvirt brings it up by itself once the modules match.
--setup-host therefore gains --reboot-if-needed, used by the deployment
profile. The reboot is scheduled through systemd-run --on-active=5 rather than
issued immediately, otherwise it would kill the installer's SSH session and the
orchestrator would report a failure for a VM that actually succeeded. Without
the flag the behaviour is unchanged: explain and exit 1, which is what a
workstation wants.
Also fix « Error setting up logfile: No write access to
/var/tmp/erplibre-virtinst/virt-manager »: that path was shared, so a first run
under sudo created it as root and later non-root runs could not write. It is
now per-UID.
Verified on the Arch VM that failed: without the flag it exits 1 with the
diagnosis; with it, the reboot is scheduled and the command still returns 0;
after the reboot the kernel and modules match, « default » is active on its own
and virbr0 exists. The cache directory is created as erplibre:erplibre 0700.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The « ERPLibre Deployment (+ QEMU + dev) » profile installed packages with a
one-liner ending in « || true » and hiding stderr, so a broken host looked
installed. Reproduced on erplibre-arch-latest: every binary was present, yet
virt-install failed and « virsh list --all » could not reach any hypervisor.
Three distinct causes, all unhandled:
- The user was never added to the libvirt group. Without it a non-root libvirt
client falls back to qemu:///session, where the « default » network does not
exist, so « --network network=default » fails while everything looks
installed. Being in the group grants the RIGHT to reach qemu:///system but
does NOT change the default URI, so virt-install and virsh now pass
« --connect qemu:///system » explicitly (new LIBVIRT_URI constant).
- dnsmasq was missing: ensure_tools only installed DAEMON_PACKAGES when the
daemon was absent, and libvirt was already there. setup_host now forces them.
- On a rolling distro, « make install_os » upgrades the kernel and the package
manager removes /lib/modules/<running kernel>. modprobe bridge then fails and
libvirt cannot create virbr0 (« Unable to create bridge virbr0: Package not
installed »). No package fixes that, only a reboot: it is now diagnosed and
reported instead of surfacing as an unreadable virsh error.
The profile now calls « deploy_qemu.py --setup-host », which reuses the
existing ensure_* chain, so package names stay defined in one place
(TOOL_PACKAGES / DAEMON_PACKAGES) and keep working for apt, dnf, pacman,
zypper and brew. It fails loudly instead of « || true ».
Verified on the Arch VM that failed: setup-host reports the stale kernel and
exits 1; after a reboot it installs dnsmasq, joins libvirt/kvm, starts the
default network, and reports « Hôte prêt » (exit 0, idempotent on rerun).
The generated command now reads « virt-install --connect qemu:///system ».
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The resume menu exposed internal key names: « Reuse database without state_4 »
tells nobody what will happen, and the most useful option -- keep the prepared
database, redo the version bumps -- was the least readable of the four.
It now prints where the migration actually stands: the zip, the database, the
target, and one line per step with its state. Step 4 reports how many version
bumps are migrated and names them, so « 0/6 · 13 14 15 16 17 18 » replaces a
list index nobody could interpret.
The four numbered options become:
[c] continue where it stopped (was: press enter)
[0-4] replay from that step (new: rewind to any step)
[n] new migration, erase everything (was: [1])
[r] keep the zip, ask everything (was: [4])
Rewinding drops « state_* » from the chosen step onward and keeps the rest:
config_* answers, the zip and the target version are decisions, not progress,
so they are no longer re-asked. state_0_search_missing_module is still forced
back to False, as the previous code did: it fills an in-memory dict the later
steps rely on. Old [2] is now « replay from 0 », old [3] « replay from 4 ».
First use of t() in this file; the new strings are translated in todo_i18n.py.
Verified against the real progression of the 12.0 -> 18.0 migration and a
synthetic one: for every step, the kept keys are exactly those of the earlier
steps, config and target survive, and the module search is forced again.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Measured on erplibre-ubuntu-2404: unattended-upgrades fired in the middle of
an Odoo 12->13 migration and restarted the PostgreSQL cluster three times
(« received fast shutdown request »). OpenUpgrade lost its connection and the
intermediate database was left half migrated.
On a development VM the ERPLibre installer now turns off unattended-upgrades
and the apt-daily timers, and drops an apt.conf.d snippet so they stay off
across reboots. dnf-automatic gets the same treatment on Fedora. It runs right
after the cloud-init wait and before the apt-get calls, so apt-daily can no
longer grab the lock between the two either -- the same contention that made
« apt-get update » fail during deployment.
Production VMs are left untouched: automatic security updates must stay on
there. The switch is the existing dev/prod answer, already threaded down to
_qemu_erplibre_remote_cmd.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When OpenUpgrade fails, the intermediate <db>_upgrade_<N> database is left
half migrated. The clone step had already been recorded as done, so a rerun
skipped it and restarted OpenUpgrade on top of that broken clone.
Clear the clone flag on failure so the replay rebuilds the intermediate
database from the pristine source. Found by a real 12.0 -> 13.0 run: Ubuntu
unattended-upgrades restarted the PostgreSQL cluster mid-migration, Odoo lost
its connection, and the replay would have resumed on the damaged clone.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
key is the only thing pairing a website copy with the module view it came
from. A copy whose key matches nothing is never paired, so it never receives
the new inherit_id, never changes shape, and never joins a view combination.
Renaming the key is therefore enough to take it out of the way:
UPDATE ir_ui_view SET key = '<prefix>.' || key, active = false
active = false alone would NOT work: an inactive copy keeping the same key
still shadows the module view. Nothing is deleted either, so inherit_id
ondelete='restrict' and the website_page foreign keys are never touched, and
the old arch stays in database as a readable archive. --restore undoes it.
Written in plain psql on purpose: it runs on a database not yet migrated,
where starting an Odoo shell of the target version is not guaranteed. It is a
systematic step driven by the detector, valid for every bump, so it lives in
the loop rather than in a per-version fix_migration file.
Offered rather than forced, since those copies carry real customizations.
Verified on a throwaway copy: dry-run writes nothing; apply renames the key and
deactivates while keeping the 4058-char arch; a second apply is a no-op;
restore returns the row to its exact initial state.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A version bump rewrites website views without announcing any of it, so "the
site looks wrong" after a migration is currently unanswerable.
snapshot_cow_views.py records every website_id view (key, mode, inherit_id,
active, arch and its md5) and diffs two snapshots. Rows come back as JSON
straight from Postgres because an arch holds newlines and pipes; the column
list is intersected with information_schema, since ir_ui_view does not expose
the same columns from 12.0 to 18.0. Snapshots hold customer template content,
so they go under private/ and stay out of git.
todo_upgrade.py takes one before and one after each OpenUpgrade run, then
prints the diff. Both are non-blocking: forensic material must never stop an
upgrade.
Measured on the real 12.0 -> 13.0 jump, the diff shows what no log reported:
71 -> 68 copies, 16 deleted and 13 created. portal.frontend_layout is not
converted but DELETED (id 2670) and RECREATED (id 3397), with its children
re-parented from one to the other. The four theme_technolibre copies,
muk_web_branding, project_agile and erplibre_website_snippets_basic_html are
dropped outright, and website_crm.contactus_thanks is renamed to
website_form.contactus_thanks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two blind spots made the detector miss real breakages.
1. Comparing « mode » is not the right test. What decides the shape an arch
must have is whether the target declares an inherit_id: with one, the arch
must be inheritance specs (<data>, <xpath>, position=); without one, it must
be a standalone template. A view moving from a root template to
« inherit_id + primary="True" » keeps mode='primary' on BOTH sides, so the
old test reported nothing while the copy still broke. The comparison is now
(target declares inherit) vs (stored arch is spec-shaped), and each finding
carries the reason. A mode change with a matching shape is still reported,
as a lesser warning.
arch_db is read as text up to 15.0 and as jsonb from 16.0; both are handled.
2. A module renamed upstream was reported as « module absent », hiding every
view it owns. renamed_modules is now read from the target OpenUpgrade
apriori.py (21 entries for 13.0, 56 for 14.0, 39 for 16.0, 20 for 18.0) and
used before concluding the module is gone.
Also correct the advice printed for a copy at risk: deactivating it is not
enough, an inactive copy keeping the same key still shadows the module view.
Renaming the key is what actually unpairs it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A copy-on-write view freezes the structure of the module view it was copied
from. When that module view changes mode between two versions, the copy keeps
an arch written for the old mode and the upgrade dies on
« Element ... cannot be located in parent view », hours into the migration.
Measured on a real 12.0 database: portal.frontend_layout is declared primary in
12.0 (full QWeb template, so a full-template arch is legitimate) and becomes an
extension in 13.0 (inherit_id + xpath). The 2021 COW copy follows the module,
turns into an extension, and keeps its 12.0 arch. The customization is genuine;
the breakage is produced by the migration.
check_cow_views.py compares the mode stored in database with the mode declared
in the target version sources and sorts the views in three buckets: at risk
(mode change), module absent from the target version, and pages made from the
editor (not at risk). Read-only, and called from step 2 so the list is known
before the long version loop rather than during it.
On the reference database it reports exactly the view that broke the upgrade,
plus 7 views whose module is gone in 13.0 (custom theme included).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Which modules must be uninstalled before a version bump depends on the data of
one specific database, so that list does not belong to a shared versioned file.
- Read private/odoo/migration/<database>/uninstall_module_list_odooXX0_to_odooYY0.txt
first, then the shared versioned defaults under script/odoo/migration/, and
merge them (duplicates dropped). private/ mirrors script/, the convention
already used by script/todo/todo.json -> private/todo/todo_override.json.
- Fix the parser. It was « f.readline().split() »: only the FIRST line was kept,
so a multi-line list was silently truncated, and a comma-separated list turned
into one bogus module name. On a file starting with a comment it returned the
words of that comment as module names. It now reads every line and accepts
commas, several names per line, blank lines and comments.
- Support a « # reason » justification per module, printed before uninstalling;
a module with no reason is flagged. Removing a module must stay reviewable.
- Ignore private/odoo/ in git: these lists describe one database.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The database migration recorded steps as done even when they had failed, so a
resume skipped them and the loop kept going on a broken database.
execute.py
- exec_command_live() left exit_code = None when an exception was raised. None
is falsy, so « if not status: » marked the step done and
« if status and wait_at_error » skipped the error prompt: a crashed command
was reported as a success. Both except blocks now set exit_code = 1.
- The « no Odoo version installed » path returned a bare -1 while callers
unpack a tuple (status, cmd), raising ValueError instead of surfacing the
failure. It now returns the same shape the caller asked for.
todo_upgrade.py
- todo_upgrade_execute(): treat a None status as a failure (defence in depth).
- Database migration (OpenUpgrade): the return code was discarded, with an
explicit « TODO detect error », and the state was written unconditionally.
It is now captured; on failure the loop stops instead of migrating the next
version on top of a half-migrated database.
- Neutralization: the flag was set before the « if not status » test, making
that test dead code. Removed the unconditional assignment.
- Clone and fix-migration hook: their return codes were never captured, so the
steps were marked done whatever happened. Both are now checked.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A leftover git daemon from an interrupted run kept port 9418, so the new
daemon failed to bind ("Address already in use") and the final « kill
$DAEMON_PID » failed on the already-dead PID, returning exit 1. That non-zero
status bubbled up through apply_extra_modules() and set exit_code=1, which
silently skipped the post-install steps (add_extra_to_config_conf,
generate_config) -> CybroOdoo never landed in config.conf.
- Kill any leftover daemon before starting a new one. Match the stable
arguments, not "git daemon": « git daemon » execs into « git-daemon »
(hyphen, /usr/lib/git-core/git-daemon), so a "git daemon" (space) pattern
never matched the actual process.
- Move daemon cleanup into an EXIT trap that tolerates an already-dead PID,
so the script never exits non-zero just because the daemon is already gone.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The extra modules (CybroOdoo) were still not applied on an already-installed
environment even after apply_extra_modules() was added. Two root causes:
1. When update_env_version.py runs as a script, sys.path[0] is
script/version/, so « from script.version.erplibre_state import ... »
raised ImportError and _STATE_AVAILABLE fell back to False. The whole
state mechanism was silently disabled: set_version_installed() never
wrote .erplibre-state.json, get_version_extra() stayed False, and the
extra manifest was never merged -> CybroOdoo never cloned. Add a sibling
import fallback so the state module also resolves under script execution.
2. generate_config.sh rewrites config.conf from a fixed addons list that
omits the extra modules. Add add_extra_to_config_conf(), called AFTER
generate_config.sh (last writer wins), which appends the extra addons
paths (derived from the "extra" group of the extra manifest) to
addons_path. Idempotent: already-present paths are skipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
« Installation avec modules extra » (--with_extra) ne clonait pas CybroOdoo
sur un env déjà installé : validate_environment renvoyait « valide » ->
update_environment (où l'état extra est posé via set_version_installed) était
sauté -> get_version_extra restait False -> git_merge_repo_manifest n'ajoutait
pas manifest/git_manifest_extra_odooXX.xml -> CybroOdoo jamais synchronisé ni
ajouté à config.conf (« Nothing to do »).
Nouvelle étape apply_extra_modules(), appelée dans main() quand --with_extra
est demandé (même si l'env de base est déjà installé) : pose l'état extra,
régénère le manifest local + repo sync (clone CybroOdoo). La post-étape qui
suit régénère config.conf, qui prend alors le nouveau chemin d'addons.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Le montage sshfs de « Configurer sshfs » ajoute désormais
« -o follow_symlinks » : sshfs résout les symlinks côté serveur. Sans ça,
git échoue sur les worktrees google-repo d'ERPLibre (leur .git est une
chaîne de symlinks relatifs profonds vers .repo/projects et project-objects)
-> « erreur à la lecture de .git », git status/commit impossibles sur le
montage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
« Code » (et Update, Git, Database…) apparaissait vide : l'extracteur AST ne
comprenait que le motif « status == "N": self.methode() » (menu QEMU). Les
menus bâtis par « choices = config_file.get_config(...) » + « choices.append(
...) » avec dispatch « str(len(choices)-N) » n'exposaient aucun enfant ->
impossible d'ouvrir leurs commandes.
Ajout de l'extraction de ce motif :
- entrées de CONFIG lues depuis todo.json (rejouées via
execute_from_configuration avec l'entrée en kwargs) ;
- entrées APPENDÉES (dict littéral OU variable menu_entry=… suivie par n° de
ligne) mappées à leur méthode via le dispatch « str(len(choices)-K) » ;
- une entrée qui ouvre un sous-menu (ex. Update) est développée récursivement.
Validé : Code -> 7 commandes (statut/remiser/formater/SHELL/màj module/débogage
+ sous-menu Update), méthodes correctes et existantes, TUI monte sans erreur.
Les dispatches complexes (ex. « Migration BD » via TodoUpgrade) restent
visibles mais non lançables (method None).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
L'entrée « Update - Update all developed staging source code » quitte le
menu Execute (renumérotation 9->8 … 15->14, dispatch elif mis à jour) et
rejoint le sous-menu « Code - Outil pour développeur » (dernière entrée,
appelle prompt_execute_update). Le libellé/i18n (🔃) est réutilisé.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
webdriver.Firefox échouait : « InvalidArgumentException: Argument
--remote-allow-system-access can't be set via capabilities ». Cet argument
(ajouté pour le Firefox snap d'Ubuntu) est désormais REFUSÉ via les
capabilities par les geckodriver récents, qui gèrent seuls l'accès système
du snap. Il cassait donc TOUS les setups sur geckodriver récent (ici Arch
sans snap). On ne le passe plus.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
extraire_svg_graphique_by_3d annote « -> Optional[str] » mais le module
n'importait pas Optional -> « NameError: name 'Optional' is not defined »
au chargement. Ajout de « from typing import Optional ».
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
« Renommer quelles VM ? » devient « Modifier quelles VM ? » : pour chaque
VM choisie, on peut changer le NOM, la taille de DISQUE et la RAM (vide =
garder). _qemu_customize_names -> _qemu_customize_vms renvoie (names,
selected) avec les valeurs mises à jour.
Le multiplicateur de ressources est désormais CUIT dans `selected` (RAM
finale) et le nombre de vCPU fixé une fois (uniforme) -> un override de RAM
par VM est une valeur ABSOLUE, sans ambiguïté avec le multiplicateur. La
fonction eff_res disparaît (plan, dry-run et déploiement lisent les valeurs
finales de `selected`).
Validé : modif VM 1 (nom/disque/RAM) appliquée, VM 2 inchangée.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- État : 10 -> 8. Le libellé « ❌ effacée » (10 cellules) forçait la largeur ;
l'état « effacée » devient l'icône seule 🗑. Reste le plus long « ⏸ pause »
(7), donc 8 suffit.
- VM : 26 -> 22 (tient les noms standards « erplibre-ubuntu-2604 » = 20 ; les
rares noms suffixés d'arch défilent via overflow-x).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
« SELinuxContext=unconfined_u:unconfined_r:unconfined_t:s0 » ne débloque PAS
l'exécution : la transition init_t -> unconfined_t est refusée par la
politique -> le service échouait toujours en 203/EXEC (« Permission denied »
sur run.sh, contexte user_home_t) sur Fedora.
En DEV (VM jetable, « SELinux relâché ») on passe désormais SELinux en
PERMISSIF (setenforce 0 + persistance dans /etc/selinux/config) si actif ->
le service exécute run.sh/venv sous /home sans blocage.
PROD reste confiné (/opt/erplibre hors user_home_t + restorecon) — à
peaufiner quand on reviendra sur la prod.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
La colonne « État » réservait 12 caractères alors que le plus long libellé
(« ❌ effacée ») en fait 10 -> elle paraissait trop large. Ramenée à 10
(sans troncature).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Le fix précédent (DPkg::Lock::Timeout) ne suffisait pas : cette option NE
couvre PAS le verrou /var/lib/apt/lists/lock de « apt-get update ». Au 1er
boot, cloud-init/apt-daily tient ce verrou -> update échouait AUSSITÔT
-> lists vides -> « Unable to locate package git » (exit 100), toujours sur
debian-12.
On RÉESSAIE désormais « apt-get update » jusqu'à libération du verrou (et
lists peuplées), borné à ~5 min (30×10s), avant l'install. Une fois update
OK, git/make s'installent normalement.
Note : le remote_cmd est construit côté HÔTE -> relancer le CLI todo pour
que la nouvelle commande soit envoyée aux VM.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Nouvelle 1re colonne « # » : numéro de séquence de chaque VM.
- Sommaire enrichi du « max » de durée : la VM TERMINÉE la plus lente
(pire cas), à côté du temps global et de l'ETA.
Validé headless : # = 1..N, sub_title « … · max MM:SS ».
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Au déploiement (« Déployer une ou plusieurs VM »), après le choix de la
branche, on demande l'ENVIRONNEMENT cible (dev par défaut / prod).
- DEV : ERPLibre dans ~/git/erplibre ; service systemd unconfined si
SELinux actif (comportement inchangé).
- PROD : ERPLibre dans /opt/erplibre (sudo git clone + chown à
l'utilisateur) ; service systemd CONFINÉ par SELinux (pas d'unconfined)
+ restorecon des contextes -> hors user_home_t, un service peut exécuter
le contenu sans lever le confinement.
Le drapeau `prod` est propagé : _qemu_deploy -> _qemu_install_erplibre_
monitored/_vm -> _qemu_erplibre_remote_cmd -> _qemu_odoo_service_cmd.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
« -v » montre l'étape mais pas la sortie des sous-processus. Seul « -vvv »
(debug) affiche git clone/checkout et le build pip -> l'erreur réelle d'une
dépendance VCS/build. Le rejeu de diagnostic passe donc en -vvv.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Corrections issues de la revue complète des 13 VM (6 en échec réel) :
- APT lock (debian-12) : au 1er boot, cloud-init/apt-daily tient le verrou
et « apt-get install » échouait aussitôt. Ajout de
« -o DPkg::Lock::Timeout=600 » (update + install) -> attend le verrou.
- wkhtmltopdf Debian 13 « trixie » (debian-13) : on prenait le build
« bullseye » (dépend de libssl1.1, absent de trixie) -> gdebi échouait.
Désormais bookworm pour bookworm/trixie/+. ET l'échec gdebi est NON
bloquant (avertissement au lieu d'exit 1 : wkhtmltopdf est optionnel,
sinon tout install_os avortait et Odoo n'était jamais installé).
- SELinux 203/EXEC (fedora-41/43/44) : un service système ne peut pas
exécuter run.sh/venv sous /home (contexte user_home_t). Ajout
conditionnel de « SELinuxContext=unconfined_u:unconfined_r:unconfined_t:s0 »
au service (uniquement si getenforce != Disabled).
- poetry status 1 (ubuntu-2604) : « -q » masquait la cause. À l'échec, on
rejoue « poetry install -v » pour capturer l'erreur dans le log.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Nouvelle colonne « Odoo » dans le dashboard : teste par TCP si l'UI web
répond sur le port 8069 de la VM (🟢 up / — sinon). Une fois « up »
détecté, on ne re-teste plus (Odoo ne redescend pas en cours d'install).
Le test (socket, timeout 0,5 s) est fait dans le thread collecteur ->
n'impacte pas la fluidité de l'UI.
Validé headless : port ouvert -> 🟢, fermé -> —.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Déploiement PARALLÈLE de 13 VM -> 2 problèmes observés :
1. « Aucune image Fedora-Cloud-Base-Generic pour Fedora 42 » alors que
l'image EXISTE. download.fedoraproject.org est un redirecteur
(MirrorManager) : sous requêtes parallèles, une VM tombait sur un miroir
incomplet -> index HTML sans correspondance -> sys.exit.
resolve_fedora_url réessaie (2× le redirecteur) puis se replie sur le
serveur MAÎTRE dl.fedoraproject.org (toujours complet). Validé : F41/F42
résolus.
2. Pavé « Fetched capabilities … » (énorme XML) dans la sortie : erreur de
LOGGING de virtinst — sous sudo, HOME/cache inaccessible -> échec
d'écriture du journal debug -> Python déverse l'enregistrement raté.
On préfixe virt-install de « env XDG_CACHE_HOME=… HOME=… » (traverse
sudo) vers un cache écrivable -> journal silencieux.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cause des GROS ralentissements en changeant de VM / lisant les logs :
- on_data_table_row_highlighted rechargeait le log à CHAQUE mouvement du
curseur (donc en rafale quand on tient la flèche) ;
- _load_selected_log(reset) lisait le fichier ENTIER (offset 0) et écrivait
TOUTES les lignes dans le RichLog, SYNCHRONE sur la boucle d'événements
-> un log de 250 Ko / plusieurs milliers de lignes gelait l'UI, × chaque
pas de navigation.
Corrections :
- _read_tail : au changement de VM on ne lit que la FIN du log (128 Ko /
1000 lignes max) et on cale l'offset sur la taille totale -> le suivi
incrémental continue depuis la fin. Affichage quasi instantané.
- Debounce (0,25 s) : RowHighlighted mémorise seulement la cible ; on ne
charge qu'une fois le curseur stabilisé, et uniquement la DERNIÈRE VM.
Validé headless : 2 flèches rapides -> 1 seul chargement (VM finale),
1000 lignes affichées au lieu de 8000.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
L'auto-refresh coupé par défaut était une mauvaise idée : (1) les lags
persistaient malgré tout ; (2) une fois coupé, on ne voyait plus
l'avancement (et l'activation ne suffisait pas). On revient au
comportement précédent : rafraîchissement automatique TOUJOURS actif
(logs 1s / table 2s / domstate 10s) — au moins on voit la progression.
Revert de 4f0c3cd. L'analyse du VRAI ralentissement (navigation / logs)
est traitée séparément.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
