mirror of
https://github.com/opnsense/src.git
synced 2026-06-12 18:20:49 -04:00
cb9d4bb1fb
The goal of this project is to integrate the relevant scripts from the FreeBSD-CI project (https://github.com/freebsd/freebsd-ci) into the src repository. This allows developers to run the test suite similar to how it is executed on ci.freebsd.org, and eventually, have it directly used by our CI system. This effort is also part of the workflow improvement project, aiming to incorporate pre-merge testing. Current Features: * Does smoke tests using either bhyve(amd64 only) or qemu(Non x86_64 or when defined USE_QEMU=1). Currently defined CITYPE=smoke. Once we have added full tests we can also utilize something like CITYPE=full * Most of the resources are dynamically allocated based on available resources in the host * If CPU supports POPCNT or vmm can be loaded then bhyve is used for amd64 otherwise automatically installs and uses qemu@nox11 * When required third party applications or packages for booting non-x86 images are automatically installed Current Limitation: * Does not support full tests like the one in our Jenkins * At this moment this is also not suitable to be used in our Jenkins platform as the jobs are divided in multiple smaller tasks and artifacts are moved here and there which are not exactly the scenario for individual developers. Future Works: * Add full tests like the one in ci.freebsd.org * Add different tests or options to disable some tests * Add test profiles full * Possibly add test through Cloud Providers like AWS/GCP/Azure or Cirrus or Github Actions * Update documentation Test Plan: cd /usr/src/tests/ci make ci make TARGET=amd64 TARGET_ARCH=amd64 ci make TARGET=amd64 TARGET_ARCH=amd64 USE_QEMU=1 ci make TARGET=arm64 TARGET_ARCH=aarch64 ci make TARGET=powerpc TARGET_ARCH=powerpc64 ci make TARGET=powerpc TARGET_ARCH=powerpc64le ci make TARGET=riscv TARGET_ARCH=riscv64 ci Reviewed by: lwhsu Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D43786 |
||
|---|---|---|
| .. | ||
| atf_python | ||
| ci | ||
| etc | ||
| examples | ||
| freebsd_test_suite | ||
| include | ||
| sys | ||
| __init__.py | ||
| conftest.py | ||
| Kyuafile | ||
| Makefile | ||
| Makefile.depend | ||
| Makefile.inc0 | ||
| README | ||
src/tests: The FreeBSD test suite
=================================
Usage of the FreeBSD test suite:
(1) Run the tests:
kyua test -k /usr/tests/Kyuafile
(2) See the test results:
kyua report
For further information on using the test suite, read tests(7):
man tests
Description of FreeBSD test suite
=================================
The build of the test suite is organized in the following manner:
* The build of all test artifacts is protected by the MK_TESTS knob.
The user can disable these with the WITHOUT_TESTS setting in
src.conf(5).
* The goal for /usr/tests/ (the installed test programs) is to follow
the same hierarchy as /usr/src/ wherever possible, which in turn drives
several of the design decisions described below. This simplifies the
discoverability of tests. We want a mapping such as:
/usr/src/bin/cp/ -> /usr/tests/bin/cp/
/usr/src/lib/libc/ -> /usr/tests/lib/libc/
/usr/src/usr.bin/cut/ -> /usr/tests/usr.bin/cut/
... and many more ...
* Test programs for specific utilities and libraries are located next
to the source code of such programs. For example, the tests for the
src/lib/libcrypt/ library live in src/lib/libcrypt/tests/. The tests/
subdirectory is optional and should, in general, be avoided.
* The src/tests/ hierarchy (this directory) provides generic test
infrastructure and glue code to join all test programs together into
a single test suite definition.
* The src/tests/ hierarchy also includes cross-functional test programs:
i.e. test programs that cover more than a single utility or library
and thus don't fit anywhere else in the tree. Consider this to follow
the same rationale as src/share/man/: this directory contains generic
manual pages while the manual pages that are specific to individual
tools or libraries live next to the source code.
In order to keep the src/tests/ hierarchy decoupled from the actual test
programs being installed --which is a worthy goal because it simplifies
the addition of new test programs and simplifies the maintenance of the
tree-- the top-level Kyuafile does not know which subdirectories may
exist upfront. Instead, such Kyuafile automatically detects, at
run-time, which */Kyuafile files exist and uses those directly.
Similarly, every directory in src/ that wants to install a Kyuafile to
just recurse into other subdirectories reuses this Kyuafile with
auto-discovery features. As an example, take a look at src/lib/tests/
whose sole purpose is to install a Kyuafile into /usr/tests/lib/.
The goal in this specific case is for /usr/tests/lib/ to be generated
entirely from src/lib/.
--