diff --git a/infrastructure/ansible/vault/README.md b/infrastructure/ansible/vault/README.md new file mode 100644 index 0000000..985809a --- /dev/null +++ b/infrastructure/ansible/vault/README.md @@ -0,0 +1,94 @@ +# Vault Ansible + +Ce répertoire contient les **secrets chiffrés** utilisés par Ansible pour l’écosystème de l’Alliance Boréale. + +## Rôle de ce répertoire + +On y place les variables sensibles, par exemple : + +- mots de passe applicatifs +- secrets PostgreSQL +- clés API +- jetons d’intégration +- secrets Keycloak / Forgejo / PowerDNS +- mots de passe de comptes techniques + +Ces fichiers sont destinés à être utilisés avec **Ansible Vault**. + +## Fichiers typiques + +Exemples : + +- `production.yml` +- `production-phase2.yml` + +Des fichiers d’exemple non sensibles peuvent aussi exister : + +- `production.yml.example` +- `production-phase2.yml.example` + +Les fichiers `*.example` servent de gabarits et **ne doivent contenir aucun vrai secret**. + +## Commandes utiles + +### Créer un nouveau fichier chiffré + +```bash +ansible-vault create vault/production.yml +``` + +### Modifier un fichier chiffré + +```bash +ansible-vault edit vault/production.yml +``` + +### Voir un fichier chiffré + +```bash +ansible-vault view vault/production.yml +``` + +### Chiffrer un fichier existant + +```bash +ansible-vault encrypt vault/production.yml +``` + +## Utilisation dans les playbooks + +Exemple : + +```bash +ansible-playbook playbooks/site.yml --ask-vault-pass +``` + +Ou avec un fichier de mot de passe : + +```bash +ansible-playbook playbooks/site.yml --vault-password-file ~/.ansible/vault-pass.txt +``` + +## Discipline minimale + +- Ne jamais committer de secret en clair. +- Ne jamais renommer un fichier `.example` en fichier réel sans le chiffrer. +- Garder les secrets regroupés par environnement ou par phase logique. +- Préférer des noms explicites. +- Éviter de mélanger secrets de prod et secrets de labo dans le même fichier. + +## Convention recommandée + +- `production.yml` : secrets communs de production +- `production-phase2.yml` : secrets propres aux services phase 2 +- autres fichiers : seulement si un découpage clair est utile + +## Rappel important + +Le dépôt peut contenir : + +- la structure, +- les exemples, +- les références de variables, + +mais **jamais les secrets en clair**. \ No newline at end of file diff --git a/infrastructure/icinga2-ansible-noc/Makefile b/infrastructure/icinga2-ansible-noc/Makefile new file mode 100644 index 0000000..b60eb00 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/Makefile @@ -0,0 +1,31 @@ +ANSIBLE ?= ansible-playbook +INVENTORY ?= ansible/inventory +PLAYBOOK ?= ansible/site.yml +LIMIT ?= +TAGS ?= + +.PHONY: help ping bootstrap deploy check validate-icinga syntax + +help: + @echo "Targets:" + @echo " make ping - tester SSH/Ansible" + @echo " make syntax - validation syntaxique Ansible" + @echo " make bootstrap - installation complète" + @echo " make deploy - rejouer le déploiement complet" + @echo " make check - checks locaux post-déploiement" + @echo " make validate-icinga - icinga2 daemon -C sur la cible" + +ping: + ansible -i $(INVENTORY) all -m ping + +syntax: + $(ANSIBLE) -i $(INVENTORY) $(PLAYBOOK) --syntax-check + +bootstrap deploy: + $(ANSIBLE) -i $(INVENTORY) $(PLAYBOOK) $(if $(LIMIT),--limit $(LIMIT),) $(if $(TAGS),--tags $(TAGS),) + +check: + ansible -i $(INVENTORY) icinga_servers -m shell -a 'systemctl is-active icinga2 icingadb icingadb-redis mariadb apache2 && icinga2 daemon -C' + +validate-icinga: + ansible -i $(INVENTORY) icinga_servers -m shell -a 'icinga2 daemon -C' diff --git a/infrastructure/icinga2-ansible-noc/README.md b/infrastructure/icinga2-ansible-noc/README.md new file mode 100644 index 0000000..d015137 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/README.md @@ -0,0 +1,84 @@ +# icinga2-ansible-noc + +Dépôt Ansible minimal et reproductible pour déployer un serveur Icinga 2 moderne sur Debian 12, avec : + +- dépôt officiel Icinga ; +- Icinga 2 ; +- Icinga DB + Redis ; +- MariaDB ; +- Icinga Web 2 ; +- module Icinga DB Web ; +- module Business Process Monitoring (BPM) ; +- génération de configuration Icinga via rôles Ansible ; +- commandes Makefile pour bootstrap, validation et déploiement. + +## Principe + +Icinga reste l'autorité de supervision technique. Le dépôt ne recode pas ce qu'Icinga sait déjà faire : checks, états, notifications, objets, groupes, templates et vues restent natifs. + +Ansible sert à produire une installation reproductible et à générer les objets de supervision depuis des variables versionnées. + +## Préparation + +```bash +cp ansible/inventory.example ansible/inventory +cp ansible/group_vars/all.yml.example ansible/group_vars/all.yml +$EDITOR ansible/inventory +$EDITOR ansible/group_vars/all.yml +``` + +## Commandes + +```bash +make ping +make bootstrap +make check +make deploy +make validate-icinga +``` + +## URL + +```text +http:///icingaweb2 +``` + +## Identifiants initiaux + +Définis dans : + +```text +ansible/group_vars/all.yml +``` + +Variables principales : + +- `icingaweb_admin_user` +- `icingaweb_admin_password` + +## Structure + +```text +ansible/ + site.yml + inventory.example + group_vars/all.yml.example + roles/ + common/ + icinga_repo/ + mariadb/ + icinga2/ + icingadb/ + icingaweb2/ + bpm/ + monitoring_config/ +Makefile +``` + +## Philosophie + +- Debian 12 vanille comme cible de départ. +- Official Icinga packages pour rester latest and greatest. +- Icinga DB au lieu de l'ancien IDO. +- Configuration générée par Ansible, déposée dans `/etc/icinga2/zones.d/global-templates/chezlepro/`. +- Validation systématique avec `icinga2 daemon -C`. diff --git a/infrastructure/icinga2-ansible-noc/ansible/ansible.cfg b/infrastructure/icinga2-ansible-noc/ansible/ansible.cfg new file mode 100644 index 0000000..80cfc4f --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/ansible.cfg @@ -0,0 +1,11 @@ +[defaults] +inventory = inventory +roles_path = roles +host_key_checking = False +retry_files_enabled = False +stdout_callback = yaml +interpreter_python = auto_silent + +[privilege_escalation] +become = True +become_method = sudo diff --git a/infrastructure/icinga2-ansible-noc/ansible/group_vars/all.yml.example b/infrastructure/icinga2-ansible-noc/ansible/group_vars/all.yml.example new file mode 100644 index 0000000..d824f4b --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/group_vars/all.yml.example @@ -0,0 +1,44 @@ +--- +timezone: America/Montreal + +icingaweb_admin_user: icingaadmin +icingaweb_admin_password: ChangeMeNow123! + +mysql_root_password: ChangeMeRoot123! +icingadb_database: icingadb +icingadb_user: icingadb +icingadb_password: ChangeMeIcingaDb123! +icingaweb_database: icingaweb2 +icingaweb_db_user: icingaweb2 +icingaweb_db_password: ChangeMeIcingaWeb123! + +icinga_api_root_password: ChangeMeApiRoot123! + +monitoring_zone_dir: /etc/icinga2/zones.d/global-templates/chezlepro + +notification_mail_to: daniel@example.test +notification_mail_from: icinga@example.test + +# Exemple volontairement simple. À remplacer par ton inventaire réel. +infrastructure_hosts: + - name: proxmox-01 + address: 192.168.12.11 + groups: [proxmox, linux] + checks: + - name: ping4 + command: hostalive + - name: ssh + command: ssh + - name: pbs-01 + address: 192.168.12.21 + groups: [backup, linux] + checks: + - name: ping4 + command: hostalive + - name: ssh + command: ssh + +hostgroups: + - proxmox + - linux + - backup diff --git a/infrastructure/icinga2-ansible-noc/ansible/inventory.example b/infrastructure/icinga2-ansible-noc/ansible/inventory.example new file mode 100644 index 0000000..1e626a6 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/inventory.example @@ -0,0 +1,2 @@ +[icinga_servers] +icinga-noc ansible_host=192.168.12.50 ansible_user=ansible diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/bpm/handlers/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/bpm/handlers/main.yml new file mode 100644 index 0000000..e69de29 diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/bpm/tasks/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/bpm/tasks/main.yml new file mode 100644 index 0000000..17c1736 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/bpm/tasks/main.yml @@ -0,0 +1,26 @@ +--- +- name: Install Icinga Web 2 Business Process module + ansible.builtin.apt: + name: icingaweb2-module-businessprocess + state: latest + +- name: Ensure BPM process directory exists + ansible.builtin.file: + path: /etc/icingaweb2/modules/businessprocess/processes + state: directory + owner: www-data + group: icingaweb2 + mode: '2770' + +- name: Enable Business Process module + ansible.builtin.command: icingacli module enable businessprocess + args: + creates: /etc/icingaweb2/enabledModules/businessprocess + +- name: Deploy starter BPM process + ansible.builtin.template: + src: chezlepro-infra.conf.j2 + dest: /etc/icingaweb2/modules/businessprocess/processes/chezlepro-infra.conf + owner: www-data + group: icingaweb2 + mode: '0660' diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/bpm/templates/chezlepro-infra.conf.j2 b/infrastructure/icinga2-ansible-noc/ansible/roles/bpm/templates/chezlepro-infra.conf.j2 new file mode 100644 index 0000000..4601d13 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/bpm/templates/chezlepro-infra.conf.j2 @@ -0,0 +1,5 @@ +# Business Process starter file generated by Ansible. +# Ajuste ensuite dans l'interface BPM si nécessaire. + +chezlepro-infrastructure = {% for host in infrastructure_hosts %}{{ host.name }};ping4{% if not loop.last %} & {% endif %}{% endfor %} +chezlepro-infrastructure.display_name = Infrastructure Chezlepro diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/common/handlers/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/common/handlers/main.yml new file mode 100644 index 0000000..e69de29 diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/common/tasks/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/common/tasks/main.yml new file mode 100644 index 0000000..c0b9ba1 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/common/tasks/main.yml @@ -0,0 +1,19 @@ +--- +- name: Set timezone + ansible.builtin.timezone: + name: "{{ timezone }}" + +- name: Install base packages + ansible.builtin.apt: + name: + - ca-certificates + - curl + - gnupg + - lsb-release + - apt-transport-https + - wget + - python3-pymysql + - python3-passlib + - sudo + state: present + update_cache: true diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icinga2/handlers/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/icinga2/handlers/main.yml new file mode 100644 index 0000000..f77f733 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icinga2/handlers/main.yml @@ -0,0 +1,5 @@ +--- +- name: restart icinga2 + ansible.builtin.systemd: + name: icinga2 + state: restarted diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icinga2/tasks/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/icinga2/tasks/main.yml new file mode 100644 index 0000000..3791180 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icinga2/tasks/main.yml @@ -0,0 +1,34 @@ +--- +- name: Install Icinga 2 and monitoring plugins + ansible.builtin.apt: + name: + - icinga2 + - monitoring-plugins + state: latest + +- name: Set up Icinga 2 API if not already configured + ansible.builtin.command: icinga2 api setup + args: + creates: /etc/icinga2/features-enabled/api.conf + notify: restart icinga2 + +- name: Configure API root user + ansible.builtin.template: + src: api-users.conf.j2 + dest: /etc/icinga2/conf.d/api-users.conf + owner: nagios + group: nagios + mode: '0640' + notify: restart icinga2 + +- name: Enable Icinga DB feature + ansible.builtin.command: icinga2 feature enable icingadb + args: + creates: /etc/icinga2/features-enabled/icingadb.conf + notify: restart icinga2 + +- name: Enable and start Icinga 2 + ansible.builtin.systemd: + name: icinga2 + enabled: true + state: started diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icinga2/templates/api-users.conf.j2 b/infrastructure/icinga2-ansible-noc/ansible/roles/icinga2/templates/api-users.conf.j2 new file mode 100644 index 0000000..7fbe7b9 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icinga2/templates/api-users.conf.j2 @@ -0,0 +1,4 @@ +object ApiUser "root" { + password = "{{ icinga_api_root_password }}" + permissions = [ "*" ] +} diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icinga_repo/handlers/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/icinga_repo/handlers/main.yml new file mode 100644 index 0000000..e69de29 diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icinga_repo/tasks/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/icinga_repo/tasks/main.yml new file mode 100644 index 0000000..99c00c9 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icinga_repo/tasks/main.yml @@ -0,0 +1,32 @@ +--- +- name: Read Debian version id + ansible.builtin.command: . /etc/os-release && echo $VERSION_ID + register: debian_version_id + changed_when: false + +- name: Read Debian codename + ansible.builtin.command: awk -F'[)(]+' '/VERSION=/ {print $2}' /etc/os-release + register: debian_codename + changed_when: false + +- name: Download Icinga archive keyring package + ansible.builtin.get_url: + url: "https://packages.icinga.com/icinga-archive-keyring_latest+debian{{ debian_version_id.stdout }}.deb" + dest: /tmp/icinga-archive-keyring.deb + mode: '0644' + +- name: Install Icinga archive keyring + ansible.builtin.apt: + deb: /tmp/icinga-archive-keyring.deb + +- name: Configure official Icinga repository + ansible.builtin.copy: + dest: "/etc/apt/sources.list.d/{{ debian_codename.stdout }}-icinga.list" + mode: '0644' + content: | + deb [signed-by=/usr/share/keyrings/icinga-archive-keyring.gpg] https://packages.icinga.com/debian icinga-{{ debian_codename.stdout }} main + deb-src [signed-by=/usr/share/keyrings/icinga-archive-keyring.gpg] https://packages.icinga.com/debian icinga-{{ debian_codename.stdout }} main + +- name: Refresh apt cache after Icinga repository setup + ansible.builtin.apt: + update_cache: true diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icingadb/handlers/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/icingadb/handlers/main.yml new file mode 100644 index 0000000..224bee9 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icingadb/handlers/main.yml @@ -0,0 +1,5 @@ +--- +- name: restart icingadb + ansible.builtin.systemd: + name: icingadb + state: restarted diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icingadb/tasks/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/icingadb/tasks/main.yml new file mode 100644 index 0000000..c599700 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icingadb/tasks/main.yml @@ -0,0 +1,39 @@ +--- +- name: Install Icinga DB components + ansible.builtin.apt: + name: + - icingadb + - icingadb-redis + - icingaweb2-module-icingadb + state: latest + +- name: Enable and start Icinga DB Redis + ansible.builtin.systemd: + name: icingadb-redis + enabled: true + state: started + +- name: Check whether Icinga DB schema is already imported + ansible.builtin.shell: "mysql --batch --skip-column-names {{ icingadb_database }} -e 'SHOW TABLES LIKE \"host\";'" + register: icingadb_schema_check + changed_when: false + failed_when: false + +- name: Import Icinga DB schema + ansible.builtin.shell: "mysql {{ icingadb_database }} < /usr/share/icingadb/schema/mysql/schema.sql" + when: icingadb_schema_check.stdout | length == 0 + +- name: Configure Icinga DB daemon + ansible.builtin.template: + src: config.yml.j2 + dest: /etc/icingadb/config.yml + owner: root + group: icingadb + mode: '0640' + notify: restart icingadb + +- name: Enable and start Icinga DB + ansible.builtin.systemd: + name: icingadb + enabled: true + state: started diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icingadb/templates/config.yml.j2 b/infrastructure/icinga2-ansible-noc/ansible/roles/icingadb/templates/config.yml.j2 new file mode 100644 index 0000000..7742203 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icingadb/templates/config.yml.j2 @@ -0,0 +1,11 @@ +database: + type: mysql + host: localhost + port: 3306 + database: {{ icingadb_database }} + user: {{ icingadb_user }} + password: {{ icingadb_password }} + +redis: + host: localhost + port: 6380 diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/handlers/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/handlers/main.yml new file mode 100644 index 0000000..e69de29 diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/tasks/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/tasks/main.yml new file mode 100644 index 0000000..8ffbfc4 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/tasks/main.yml @@ -0,0 +1,96 @@ +--- +- name: Install Apache, PHP and Icinga Web 2 + ansible.builtin.apt: + name: + - apache2 + - libapache2-mod-php + - php + - php-cli + - php-intl + - php-mysql + - php-gd + - php-curl + - php-mbstring + - php-xml + - icingaweb2 + - icingacli + state: latest + +- name: Ensure Icinga Web 2 config directories exist + ansible.builtin.file: + path: "{{ item }}" + state: directory + owner: www-data + group: icingaweb2 + mode: '2770' + loop: + - /etc/icingaweb2 + - /etc/icingaweb2/modules + - /etc/icingaweb2/modules/icingadb + - /etc/icingaweb2/modules/icingadb/config + +- name: Configure Icinga Web resources + ansible.builtin.template: + src: resources.ini.j2 + dest: /etc/icingaweb2/resources.ini + owner: www-data + group: icingaweb2 + mode: '0660' + +- name: Configure Icinga Web authentication + ansible.builtin.template: + src: authentication.ini.j2 + dest: /etc/icingaweb2/authentication.ini + owner: www-data + group: icingaweb2 + mode: '0660' + +- name: Configure Icinga Web roles + ansible.builtin.template: + src: roles.ini.j2 + dest: /etc/icingaweb2/roles.ini + owner: www-data + group: icingaweb2 + mode: '0660' + +- name: Create admin user password hash + ansible.builtin.command: "openssl passwd -1 {{ icingaweb_admin_password }}" + register: icingaweb_admin_hash + changed_when: false + no_log: true + +- name: Configure local Icinga Web users + ansible.builtin.template: + src: users.ini.j2 + dest: /etc/icingaweb2/users.ini + owner: www-data + group: icingaweb2 + mode: '0660' + no_log: true + +- name: Configure Icinga DB Web module database + ansible.builtin.template: + src: icingadb-config.ini.j2 + dest: /etc/icingaweb2/modules/icingadb/config.ini + owner: www-data + group: icingaweb2 + mode: '0660' + +- name: Configure Icinga DB Web command transport + ansible.builtin.template: + src: icingadb-commandtransports.ini.j2 + dest: /etc/icingaweb2/modules/icingadb/commandtransports.ini + owner: www-data + group: icingaweb2 + mode: '0660' + +- name: Enable Icinga DB Web module + ansible.builtin.command: icingacli module enable icingadb + args: + creates: /etc/icingaweb2/enabledModules/icingadb + +- name: Enable and start Apache + ansible.builtin.systemd: + name: apache2 + enabled: true + state: started diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/authentication.ini.j2 b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/authentication.ini.j2 new file mode 100644 index 0000000..d0089f8 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/authentication.ini.j2 @@ -0,0 +1,2 @@ +[icingaweb2] +backend = "ini" diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/icingadb-commandtransports.ini.j2 b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/icingadb-commandtransports.ini.j2 new file mode 100644 index 0000000..5987d34 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/icingadb-commandtransports.ini.j2 @@ -0,0 +1,6 @@ +[icinga2] +transport = "api" +host = "localhost" +port = "5665" +username = "root" +password = "{{ icinga_api_root_password }}" diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/icingadb-config.ini.j2 b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/icingadb-config.ini.j2 new file mode 100644 index 0000000..a9d2814 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/icingadb-config.ini.j2 @@ -0,0 +1,2 @@ +[icingadb] +resource = "icingadb" diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/resources.ini.j2 b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/resources.ini.j2 new file mode 100644 index 0000000..6b98a5d --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/resources.ini.j2 @@ -0,0 +1,19 @@ +[icingaweb_db] +type = "db" +db = "mysql" +host = "localhost" +port = "3306" +dbname = "{{ icingaweb_database }}" +username = "{{ icingaweb_db_user }}" +password = "{{ icingaweb_db_password }}" +charset = "utf8mb4" + +[icingadb] +type = "db" +db = "mysql" +host = "localhost" +port = "3306" +dbname = "{{ icingadb_database }}" +username = "{{ icingadb_user }}" +password = "{{ icingadb_password }}" +charset = "utf8mb4" diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/roles.ini.j2 b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/roles.ini.j2 new file mode 100644 index 0000000..63fd316 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/roles.ini.j2 @@ -0,0 +1,3 @@ +[Administrators] +users = "{{ icingaweb_admin_user }}" +permissions = "*" diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/users.ini.j2 b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/users.ini.j2 new file mode 100644 index 0000000..ea0eece --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/icingaweb2/templates/users.ini.j2 @@ -0,0 +1,3 @@ +[{{ icingaweb_admin_user }}] +password = "{{ icingaweb_admin_hash.stdout }}" +active = "1" diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/mariadb/handlers/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/mariadb/handlers/main.yml new file mode 100644 index 0000000..e69de29 diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/mariadb/tasks/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/mariadb/tasks/main.yml new file mode 100644 index 0000000..8be25f0 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/mariadb/tasks/main.yml @@ -0,0 +1,43 @@ +--- +- name: Install MariaDB + ansible.builtin.apt: + name: + - mariadb-server + - mariadb-client + state: present + +- name: Enable and start MariaDB + ansible.builtin.systemd: + name: mariadb + enabled: true + state: started + +- name: Create Icinga DB database + community.mysql.mysql_db: + name: "{{ icingadb_database }}" + state: present + login_unix_socket: /run/mysqld/mysqld.sock + +- name: Create Icinga DB user + community.mysql.mysql_user: + name: "{{ icingadb_user }}" + password: "{{ icingadb_password }}" + priv: "{{ icingadb_database }}.*:ALL" + host: localhost + state: present + login_unix_socket: /run/mysqld/mysqld.sock + +- name: Create Icinga Web database + community.mysql.mysql_db: + name: "{{ icingaweb_database }}" + state: present + login_unix_socket: /run/mysqld/mysqld.sock + +- name: Create Icinga Web database user + community.mysql.mysql_user: + name: "{{ icingaweb_db_user }}" + password: "{{ icingaweb_db_password }}" + priv: "{{ icingaweb_database }}.*:ALL" + host: localhost + state: present + login_unix_socket: /run/mysqld/mysqld.sock diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/handlers/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/handlers/main.yml new file mode 100644 index 0000000..2116acf --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/handlers/main.yml @@ -0,0 +1,3 @@ +--- +- name: validate and restart icinga2 + ansible.builtin.shell: icinga2 daemon -C && systemctl restart icinga2 diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/tasks/main.yml b/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/tasks/main.yml new file mode 100644 index 0000000..db51b26 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/tasks/main.yml @@ -0,0 +1,35 @@ +--- +- name: Ensure generated Icinga config directory exists + ansible.builtin.file: + path: "{{ monitoring_zone_dir }}" + state: directory + owner: nagios + group: nagios + mode: '0750' + +- name: Generate hostgroups + ansible.builtin.template: + src: hostgroups.conf.j2 + dest: "{{ monitoring_zone_dir }}/hostgroups.conf" + owner: nagios + group: nagios + mode: '0640' + notify: validate and restart icinga2 + +- name: Generate hosts + ansible.builtin.template: + src: hosts.conf.j2 + dest: "{{ monitoring_zone_dir }}/hosts.conf" + owner: nagios + group: nagios + mode: '0640' + notify: validate and restart icinga2 + +- name: Generate services + ansible.builtin.template: + src: services.conf.j2 + dest: "{{ monitoring_zone_dir }}/services.conf" + owner: nagios + group: nagios + mode: '0640' + notify: validate and restart icinga2 diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/templates/hostgroups.conf.j2 b/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/templates/hostgroups.conf.j2 new file mode 100644 index 0000000..aca0f6a --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/templates/hostgroups.conf.j2 @@ -0,0 +1,5 @@ +{% for group in hostgroups %} +object HostGroup "{{ group }}" { + display_name = "{{ group }}" +} +{% endfor %} diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/templates/hosts.conf.j2 b/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/templates/hosts.conf.j2 new file mode 100644 index 0000000..f956c37 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/templates/hosts.conf.j2 @@ -0,0 +1,8 @@ +{% for host in infrastructure_hosts %} +object Host "{{ host.name }}" { + import "generic-host" + address = "{{ host.address }}" + vars.os = "Linux" + groups = [ {% for group in host.groups | default([]) %}"{{ group }}"{% if not loop.last %}, {% endif %}{% endfor %} ] +} +{% endfor %} diff --git a/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/templates/services.conf.j2 b/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/templates/services.conf.j2 new file mode 100644 index 0000000..fdb0ca6 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/roles/monitoring_config/templates/services.conf.j2 @@ -0,0 +1,9 @@ +{% for host in infrastructure_hosts %} +{% for check in host.checks | default([]) %} +apply Service "{{ check.name }}" { + import "generic-service" + check_command = "{{ check.command }}" + assign where host.name == "{{ host.name }}" +} +{% endfor %} +{% endfor %} diff --git a/infrastructure/icinga2-ansible-noc/ansible/site.yml b/infrastructure/icinga2-ansible-noc/ansible/site.yml new file mode 100644 index 0000000..eb0a9e2 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/ansible/site.yml @@ -0,0 +1,13 @@ +--- +- name: Deploy Icinga 2 NOC server + hosts: icinga_servers + become: true + roles: + - common + - icinga_repo + - mariadb + - icinga2 + - icingadb + - icingaweb2 + - bpm + - monitoring_config diff --git a/infrastructure/icinga2-ansible-noc/docs/ARCHITECTURE.md b/infrastructure/icinga2-ansible-noc/docs/ARCHITECTURE.md new file mode 100644 index 0000000..6305050 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/docs/ARCHITECTURE.md @@ -0,0 +1,50 @@ +# Architecture + +## Cible + +Ce dépôt installe une pile Icinga moderne sur Debian 12 : + +```text +Icinga 2 -> Icinga DB feature -> Redis -> Icinga DB daemon -> MariaDB -> Icinga Web 2 + -> BPM module +``` + +## Choix structurants + +- Icinga 2 demeure le moteur de supervision. +- Icinga DB remplace l'ancien backend IDO pour rester aligné avec la pile moderne. +- Icinga Web 2 fournit l'interface opérateur. +- BPM sert à représenter des regroupements métier ou opérationnels. +- Ansible génère les objets Icinga depuis `group_vars/all.yml`. + +## Emplacement des configurations générées + +```text +/etc/icinga2/zones.d/global-templates/chezlepro/ +``` + +Fichiers générés : + +- `hostgroups.conf` +- `hosts.conf` +- `services.conf` + +## Cycle opératoire + +```bash +make deploy +make validate-icinga +make check +``` + +## Extension prévue + +Ajouter progressivement : + +- templates SNMP ; +- checks Proxmox ; +- checks Ceph ; +- checks PBS ; +- checks HTTP/TLS ; +- notifications mail ; +- intégration agent Icinga sur les hôtes Linux. diff --git a/infrastructure/icinga2-ansible-noc/docs/RUNBOOK.md b/infrastructure/icinga2-ansible-noc/docs/RUNBOOK.md new file mode 100644 index 0000000..0079a91 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/docs/RUNBOOK.md @@ -0,0 +1,50 @@ +# Runbook + +## Premier déploiement + +```bash +ansible-galaxy collection install -r requirements.yml +cp ansible/inventory.example ansible/inventory +cp ansible/group_vars/all.yml.example ansible/group_vars/all.yml +$EDITOR ansible/inventory +$EDITOR ansible/group_vars/all.yml +make bootstrap +``` + +## Validation + +```bash +make validate-icinga +make check +``` + +## Ajouter un hôte + +Modifier `ansible/group_vars/all.yml` : + +```yaml +infrastructure_hosts: + - name: nouveau-serveur + address: 192.168.12.99 + groups: [linux] + checks: + - name: ping4 + command: hostalive + - name: ssh + command: ssh +``` + +Puis : + +```bash +make deploy +``` + +## Dépannage rapide + +```bash +sudo systemctl status icinga2 icingadb icingadb-redis mariadb apache2 --no-pager +sudo icinga2 daemon -C +sudo journalctl -u icinga2 -n 100 --no-pager +sudo journalctl -u icingadb -n 100 --no-pager +``` diff --git a/infrastructure/icinga2-ansible-noc/requirements.yml b/infrastructure/icinga2-ansible-noc/requirements.yml new file mode 100644 index 0000000..6a0d242 --- /dev/null +++ b/infrastructure/icinga2-ansible-noc/requirements.yml @@ -0,0 +1,3 @@ +--- +collections: + - name: community.mysql diff --git a/post-mortems/post-mortem-vishnu-freeze-am5.md b/post-mortems/post-mortem-vishnu-freeze-am5.md new file mode 100644 index 0000000..65a5ade --- /dev/null +++ b/post-mortems/post-mortem-vishnu-freeze-am5.md @@ -0,0 +1,201 @@ +# Post-mortem : freeze hard d'un nœud Proxmox AM5 — diagnostic et mitigation + +## TL;DR + +Un nœud Proxmox/Ceph monté sur ASUS TUF X670E-Plus (Ryzen AM5) gelait sans trace, sans panic, sans rien dans les logs — reset manuel obligatoire. Cause : bug C-states profonds bien documenté sur AM5 sous Linux. Mitigation immédiate via `processor.max_cstate=2` au cmdline kernel. Correction durable : flash BIOS vers une AGESA récente. Bonus : découverte d'une install Proxmox EFI avec `grub-pc` à la place de `grub-efi-amd64` — silencieusement bancale depuis l'origine. + +## Contexte + +- **Cluster** : 3 nœuds Proxmox/Ceph (gandalf, asgard, vishnu) +- **Nœud problématique** : vishnu — ASUS TUF Gaming X670E-Plus WiFi, BIOS 3602, AMD Raphael/Granite Ridge, kernel `6.8.12-20-pve` +- **Particularité** : seul nœud avec passthrough USB (stick FTDI 0403:6015 vers VM Home Assistant) +- **Storage local** : 1 OSD HDD 9.1 TiB (LUKS), 1 OSD NVMe 1.8 TiB, NVMe 100 GiB pour DB +- **Symptôme** : freeze hard récurrent, écran figé, plus aucune réponse réseau ni console, reset hardware obligatoire + +## Symptômes — ce qui rend le cas difficile + +Le tableau clinique éliminait d'emblée plusieurs pistes classiques : + +- Aucun kernel panic à l'écran ou en pstore → pas de panic propre +- Aucun message dans `journalctl -k -b -1 | tail -100` avant la coupure → kernel n'a pas eu le temps d'écrire +- Le softdog watchdog présent (`soft_margin=60`) n'a jamais déclenché de reboot automatique → pas un soft lockup détectable +- Pas de redémarrage seul, intervention manuelle obligatoire +- HEALTH_WARN Ceph avec slow ops BlueStore — mais sur osd.4 et osd.5 (situés sur les **autres** nœuds), pas sur osd.3 (vishnu). Donc Ceph subissait le freeze, n'en était pas la cause. + +Bref : machine qui meurt instantanément, sans signal préalable. C'est le profil typique d'un blocage hardware ou d'un deadlock kernel total où plus aucune IRQ ne remonte. + +## Hypothèses écartées en cours de diag + +| Hypothèse | Pourquoi écartée | +|-----------|------------------| +| Disque OSD mourant (slow ops BlueStore) | Slow ops sur OSD distants, pas sur celui de vishnu — symptôme, pas cause | +| Quorum corosync perdu | Logs corosync sains, quorum stable, MTU PMTUD négocié à 1397 | +| OOM / pression mémoire | Aucun message OOM dans les logs | +| MCE matérielle | Aucune entrée MCE décodée par le kernel | +| Bug VFIO / IOMMU groups sales | Vérifié — pas de PCI passthrough actif (uniquement USB par vendor:product) | + +## L'indice qui a tout débloqué + +Trois faits convergents : + +1. **Hardware AM5 récent** (Ryzen 7000/9000 sur X670E) +2. **Cmdline kernel nu** : `BOOT_IMAGE=/boot/vmlinuz-6.8.12-20-pve root=/dev/mapper/pve-root ro quiet` — aucune mitigation, aucun paramètre IOMMU, aucun ajustement idle +3. **Profil de freeze** : hard hang sans trace, propre à un seul nœud + +Cette combinaison correspond à un bug largement documenté de l'écosystème AM5 sous Linux : sous certaines conditions de charge, le CPU descend dans un état d'idle profond (C3/C6) duquel il ne se réveille pas correctement à l'arrivée d'une IRQ. Le core est physiquement gelé — pas de fenêtre pour écrire un panic, pour qu'un watchdog software se déclenche, ou pour que la console réagisse. + +C'est exactement ce que vishnu manifestait. + +## La mitigation appliquée + +### 1. Modification du cmdline kernel + +```bash +# Backup +cp /etc/default/grub /etc/default/grub.bak.$(date +%Y%m%d) + +# Édition +sed -i 's|^GRUB_CMDLINE_LINUX_DEFAULT=.*|GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt processor.max_cstate=2"|' /etc/default/grub +``` + +Trois paramètres : +- `processor.max_cstate=2` — la mitigation principale, bloque l'accès aux C-states ≥ C3 +- `amd_iommu=on iommu=pt` — passage du mode DMA "Translated lazy" (défaut) au mode passthrough optimisé, recommandé dès qu'il y a virtualisation + +### 2. Filets de sécurité pour les prochaines fois + +```bash +cat > /etc/sysctl.d/99-debug-freeze.conf </dev/null +ls -la /boot/efi/EFI/ + +# Boot du dernier crash +journalctl -k -b -1 | tail -100 +journalctl -b -1 -p err +``` + +## Références utiles + +- Proxmox Wiki — PCI(e) Passthrough : https://pve.proxmox.com/wiki/PCI(e)_Passthrough +- Proxmox Wiki — Host Bootloader : https://pve.proxmox.com/wiki/Host_Bootloader +- AMD AGESA changelog (par modèle de carte mère, sur le site du fabricant) — vérifier régulièrement les mentions "stability" + +--- + +*Post-mortem rédigé suite à un diagnostic du 29 avril 2026. Hardware : ASUS TUF Gaming X670E-Plus WiFi, BIOS 3602. Stack : Proxmox VE 8 / kernel 6.8.12-20-pve / Ceph Quincy.*