upstream/bind9
1
0
Fork 0
mirror of https://github.com/isc-projects/bind9.git synced 2026-04-15 22:09:31 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

chg: ci: Revise merge request pipeline job triggering rules

Browse source 
Over the past few years, some of the initial decisions made about which
GitLab CI jobs to run for all merge requests and which of them to run
just for scheduled/web-triggered pipelines turned out to be less than
ideal in practice: test coverage was found to be too lax in some areas
and on the other hand unnecessarily repetitive in others.  For example,
compilation failures for certain build types that are not exercised for
every merge request (e.g. FIPS-enabled builds) turned out to be much
more common in practice than e.g. test failures happening only on a
subset of releases of a given Linux distribution.

To limit excessive resource use while retaining broad test coverage,
adjust GitLab CI job triggering rules for merge request pipelines as
follows:

- run all possible build jobs for every merge request; compilation
failures triggered for build flavors that were only tested in
scheduled pipelines turned out to be surprisingly commonplace and
became a nuisance over time, particularly given that the run times
of build jobs are much lower than those of test jobs,

- for every merge request, run at least one system & unit test job for
each build flavor (e.g. sanitizer-enabled, FIPS-enabled,
out-of-tree, tarball-based, etc.),

- limit the amount of test jobs run for each distinct operating
system; for example, only run system & unit test jobs for Ubuntu
24.04 Noble Numbat in merge request pipelines, skipping those for
Ubuntu 22.04 Jammy Jellyfish and Ubuntu 20.04 Focal Fossa (while
still running them in other pipeline types, e.g. in scheduled
pipelines),

- ensure every merge request is tested on Oracle Linux 8, which is the
operating system with the oldest package versions out of the systems
that are still supported by this BIND 9 branch,

- decrease the number of test jobs run with sanitizers enabled while
still testing with both ASAN and TSAN and both GCC and Clang for
every merge request.

These changes do not affect the set of jobs created for any other
pipeline type (triggered by a schedule, by a GitLab API call, by the web
interface, etc.); only merge request pipelines are affected.

---

Since understanding the impact of this MR just by looking at the diff is
arguably challenging, I prepared some tables showing which jobs are
currently triggered for every merge request and what the new state of
things will be after this MR gets merged.

**Legend:**

  - 📈 - job was *not* run for every merge
    request before, but will be

  - 📉 - job was run for every merge request
    before, but will *not* be any longer

| Change | Job | Stage | Before | After | cff39d32455 | 2f1995c7136 / 4ad8c86cf2 |
| ------ | --- | ----- | ------ | ----- | ----------- | ----------- |
| | `docs` |  `docs` | ✅ | ✅ | ✅ | ✅ |
| | `docs:tarball` |  `docs` | ✅ | ✅ | ✅ | ✅ |
| | `clang:asan` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `clang:bookworm:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `clang:freebsd13:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `clang:freebsd14:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `clang:openbsd:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `clang:tsan` |  `build` | ✅ | ✅ | ✅ | ✅ |
| 📈| `gcc:8fips:amd64` |  `build` | ❌ | ✅ | ✅ | ✅ |
| 📈| `gcc:9fips:amd64` |  `build` | ❌ | ✅ | ✅ | ✅ |
| | `gcc:alpine3.21:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:asan` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:bookworm:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:bookworm:amd64cross32` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:bookworm:rbt:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:focal:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:jammy:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:noble:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:oraclelinux8:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:oraclelinux9:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:ossl3:sid:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:out-of-tree` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:sid:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:tarball` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:tarball:nosphinx` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:tsan` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `gcc:tumbleweed:amd64` |  `build` | ✅ | ✅ | ✅ | ✅ |
| | `cross-version-config-tests` |  `system` | ✅ | ✅ | ✅ | ✅ |
| | `respdiff` |  `system` | ✅ | ✅ | ✅ | ✅ |
| | `respdiff-third-party` |  `system` | ✅ | ✅ | ✅ | ✅ |
| | `respdiff:asan` |  `system` | ✅ | ✅ | ✅ | ✅ |
| | `respdiff:tsan` |  `system` | ✅ | ✅ | ✅ | ✅ |
| 📉| `system:clang:asan` |  `system` | ✅ | ✅ | ✅ | ❌ |
| 📉| `system:clang:bookworm:amd64` |  `system` | ✅ | ❌ | ❌ | ❌ |
| 📉| `system:clang:freebsd13:amd64` |  `system` | ✅ | ❌ | ❌ | ❌ |
| | `system:clang:freebsd14:amd64` |  `system` | ✅ | ✅ | ✅ | ✅ |
| | `system:clang:tsan` |  `system` | ✅ | ✅ | ✅ | ✅ |
| 📈| `system:gcc:8fips:amd64` |  `system` | ❌ | ❌ | ✅ | ✅ |
| | `system:gcc:9fips:amd64` |  `system` | ❌ | ✅ | ❌ | ❌ |
| | `system:gcc:alpine3.21:amd64` |  `system` | ✅ | ✅ | ✅ | ✅ |
| | `system:gcc:asan` |  `system` | ✅ | ❌ | ❌ | ✅ |
| | `system:gcc:bookworm:amd64` |  `system` | ✅ | ✅ | ✅ | ✅ |
| | `system:gcc:bookworm:rbt:amd64` |  `system` | ✅ | ✅ | ✅ | ✅ |
| 📉| `system:gcc:focal:amd64` |  `system` | ✅ | ❌ | ❌ | ❌ |
| 📉| `system:gcc:jammy:amd64` |  `system` | ✅ | ❌ | ❌ | ❌ |
| | `system:gcc:noble:amd64` |  `system` | ✅ | ✅ | ✅ | ✅ |
| 📉| `system:gcc:oraclelinux8:amd64` |  `system` | ✅ | ❌ | ❌ | ❌ |
| | `system:gcc:oraclelinux9:amd64` |  `system` | ✅ | ✅ | ✅ | ✅ |
| | `system:gcc:ossl3:sid:amd64` |  `system` | ✅ | ✅ | ✅ | ✅ |
| 📈| `system:gcc:out-of-tree` |  `system` | ❌ | ✅ | ✅ | ✅ |
| 📉| `system:gcc:sid:amd64` |  `system` | ✅ | ❌ | ❌ | ❌ |
| 📈| `system:gcc:tarball` |  `system` | ❌ | ✅ | ✅ | ✅ |
| 📉| `system:gcc:tsan` |  `system` | ✅ | ❌ | ❌ | ❌ |
| | `system:gcc:tumbleweed:amd64` |  `system` | ✅ | ✅ | ✅ | ✅ |
| 📉| `unit:clang:asan` |  `unit` | ✅ | ✅ | ✅ | ❌ |
| 📉| `unit:clang:bookworm:amd64` |  `unit` | ✅ | ❌ | ❌ | ❌ |
| 📉| `unit:clang:freebsd13:amd64` |  `unit` | ✅ | ❌ | ❌ | ❌ |
| | `unit:clang:freebsd14:amd64` |  `unit` | ✅ | ✅ | ✅ | ✅ |
| | `unit:clang:openbsd:amd64` |  `unit` | ✅ | ✅ | ✅ | ✅ |
| | `unit:clang:tsan` |  `unit` | ✅ | ✅ | ✅ | ✅ |
| 📈| `unit:gcc:8fips:amd64` |  `unit` | ❌ | ❌ | ✅ | ✅ |
| | `unit:gcc:9fips:amd64` |  `unit` | ❌ | ✅ | ❌ | ❌ |
| | `unit:gcc:alpine3.21:amd64` |  `unit` | ✅ | ✅ | ✅ | ✅ |
| | `unit:gcc:asan` |  `unit` | ✅ | ❌ | ❌ | ✅ |
| | `unit:gcc:bookworm:amd64` |  `unit` | ✅ | ✅ | ✅ | ✅ |
| | `unit:gcc:bookworm:rbt:amd64` |  `unit` | ✅ | ✅ | ✅ | ✅ |
| 📉| `unit:gcc:focal:amd64` |  `unit` | ✅ | ❌ | ❌ | ❌ |
| 📉| `unit:gcc:jammy:amd64` |  `unit` | ✅ | ❌ | ❌ | ❌ |
| | `unit:gcc:noble:amd64` |  `unit` | ✅ | ✅ | ✅ | ✅ |
| 📉| `unit:gcc:oraclelinux8:amd64` |  `unit` | ✅ | ❌ | ❌ | ❌ |
| | `unit:gcc:oraclelinux9:amd64` |  `unit` | ✅ | ✅ | ✅ | ✅ |
| | `unit:gcc:ossl3:amd64` |  `unit` | ✅ | ✅ | ✅ | ✅ |
| 📈| `unit:gcc:out-of-tree` |  `unit` | ❌ | ✅ | ✅ | ✅ |
| 📉| `unit:gcc:sid:amd64` |  `unit` | ✅ | ❌ | ❌ | ❌ |
| 📈| `unit:gcc:tarball` |  `unit` | ❌ | ✅ | ✅ | ✅ |
| 📉| `unit:gcc:tsan` |  `unit` | ✅ | ❌ | ❌ | ❌ |
| | `unit:gcc:tumbleweed:amd64` |  `unit` | ✅ | ✅ | ✅ | ✅ |

And a short statistical summary of the changes proposed:

| Stage | Before | After | Diff |
| ----- | ------ | ----- | ---- |
| `docs` | 2 | 2 | **0** |
| `build` | 23 | 25 | **+2** |
| `system` | 23 | 18 | **-5** |
| `unit` | 19 | 14 | **-5** |
| **TOTAL** | **67** | **59** | **-8** |

Mattermost thread (sparked by @pspacek):
https://mattermost.isc.org/isc/pl/z6nymnu4m3dhzr3rxtjkzzgk7a

Merge branch 'michal/revise-ci-job-triggering-rules' into 'main'

See merge request isc-projects/bind9!10349
This commit is contained in:
Michał Kępień 2025-04-29 16:19:36 +00:00
commit b4b00748bc
1 changed files with 17 additions and 9 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

26
.gitlab-ci.yml
View file

@ -840,6 +840,7 @@ gcc:oraclelinux8:amd64:
system:gcc:oraclelinux8:amd64:
<<: *oraclelinux_8_amd64_image
<<: *system_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: gcc:oraclelinux8:amd64
artifacts: true
@ -847,6 +848,7 @@ system:gcc:oraclelinux8:amd64:
unit:gcc:oraclelinux8:amd64:
<<: *oraclelinux_8_amd64_image
<<: *unit_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: gcc:oraclelinux8:amd64
artifacts: true
@ -884,12 +886,10 @@ gcc:8fips:amd64:
EXTRA_CONFIGURE: "--with-libidn2 --enable-fips-mode --disable-tracing"
<<: *oraclelinux_8fips_amd64_image
<<: *build_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
system:gcc:8fips:amd64:
<<: *oraclelinux_8fips_amd64_image
<<: *system_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: gcc:8fips:amd64
artifacts: true
@ -897,7 +897,6 @@ system:gcc:8fips:amd64:
unit:gcc:8fips:amd64:
<<: *oraclelinux_8fips_amd64_image
<<: *unit_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: gcc:8fips:amd64
artifacts: true
@ -909,7 +908,6 @@ gcc:9fips:amd64:
EXTRA_CONFIGURE: "--with-libidn2 --enable-fips-mode --disable-leak-detection --disable-tracing"
<<: *oraclelinux_9fips_amd64_image
<<: *build_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
system:gcc:9fips:amd64:
<<: *oraclelinux_9fips_amd64_image
@ -1085,7 +1083,7 @@ gcc:sid:amd64:
system:gcc:sid:amd64:
<<: *debian_sid_amd64_image
<<: *system_test_job
<<: *system_test_make_check_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: gcc:sid:amd64
artifacts: true
@ -1093,6 +1091,7 @@ system:gcc:sid:amd64:
unit:gcc:sid:amd64:
<<: *debian_sid_amd64_image
<<: *unit_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: gcc:sid:amd64
artifacts: true
@ -1119,7 +1118,6 @@ system:gcc:out-of-tree:
artifacts: true
<<: *base_image
<<: *system_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
unit:gcc:out-of-tree:
variables:
@ -1129,7 +1127,6 @@ unit:gcc:out-of-tree:
artifacts: true
<<: *base_image
<<: *unit_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
# Jobs for tarball GCC builds on Debian 12 "bookworm" (amd64)
@ -1149,7 +1146,6 @@ gcc:tarball:
system:gcc:tarball:
<<: *base_image
<<: *system_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
before_script:
- cd bind-*
- *setup_interfaces
@ -1163,7 +1159,6 @@ system:gcc:tarball:
unit:gcc:tarball:
<<: *base_image
<<: *unit_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
before_script:
- cd bind-*
needs:
@ -1183,6 +1178,7 @@ gcc:tumbleweed:amd64:
system:gcc:tumbleweed:amd64:
<<: *tumbleweed_latest_amd64_image
<<: *system_test_job
<<: *system_test_make_check_job
needs:
- job: gcc:tumbleweed:amd64
artifacts: true
@ -1207,6 +1203,7 @@ gcc:focal:amd64:
system:gcc:focal:amd64:
<<: *ubuntu_focal_amd64_image
<<: *system_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: gcc:focal:amd64
artifacts: true
@ -1214,6 +1211,7 @@ system:gcc:focal:amd64:
unit:gcc:focal:amd64:
<<: *ubuntu_focal_amd64_image
<<: *unit_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: gcc:focal:amd64
artifacts: true
@ -1231,6 +1229,7 @@ gcc:jammy:amd64:
system:gcc:jammy:amd64:
<<: *ubuntu_jammy_amd64_image
<<: *system_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: gcc:jammy:amd64
artifacts: true
@ -1238,6 +1237,7 @@ system:gcc:jammy:amd64:
unit:gcc:jammy:amd64:
<<: *ubuntu_jammy_amd64_image
<<: *unit_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: gcc:jammy:amd64
artifacts: true
@ -1307,6 +1307,7 @@ system:clang:asan:
LSAN_OPTIONS: "suppressions=$CI_PROJECT_DIR/suppr-lsan.txt"
<<: *base_image
<<: *system_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: clang:asan
artifacts: true
@ -1314,6 +1315,7 @@ system:clang:asan:
unit:clang:asan:
<<: *base_image
<<: *unit_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: clang:asan
artifacts: true
@ -1334,6 +1336,7 @@ system:gcc:tsan:
TSAN_OPTIONS: "${TSAN_OPTIONS_FEDORA}"
<<: *tsan_fedora_42_amd64_image
<<: *system_test_tsan_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: gcc:tsan
artifacts: true
@ -1343,6 +1346,7 @@ unit:gcc:tsan:
TSAN_OPTIONS: "${TSAN_OPTIONS_FEDORA}"
<<: *tsan_fedora_42_amd64_image
<<: *unit_test_tsan_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: gcc:tsan
artifacts: true
@ -1419,6 +1423,7 @@ clang:bookworm:amd64:
system:clang:bookworm:amd64:
<<: *debian_bookworm_amd64_image
<<: *system_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: clang:bookworm:amd64
artifacts: true
@ -1426,6 +1431,7 @@ system:clang:bookworm:amd64:
unit:clang:bookworm:amd64:
<<: *debian_bookworm_amd64_image
<<: *unit_test_job
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: clang:bookworm:amd64
artifacts: true
@ -1439,6 +1445,7 @@ clang:freebsd13:amd64:
system:clang:freebsd13:amd64:
<<: *system_test_job
<<: *freebsd_autoscaler_13_amd64
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: clang:freebsd13:amd64
artifacts: true
@ -1446,6 +1453,7 @@ system:clang:freebsd13:amd64:
unit:clang:freebsd13:amd64:
<<: *unit_test_job
<<: *freebsd_autoscaler_13_amd64
<<: *api_pipelines_schedules_tags_triggers_web_triggering_rules
needs:
- job: clang:freebsd13:amd64
artifacts: true