upstream/opnsense-src
1
0
Fork 0
mirror of https://github.com/opnsense/src.git synced 2026-03-14 22:53:16 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
opnsense-src/bin/sh/tests/execution/Makefile

74 lines
1.9 KiB
Makefile
Raw Normal View History

Migrate tools/regression/bin/ tests to the new layout. This change is a proof of concept on how to easily integrate existing tests from the tools/regression/ hierarchy into the /usr/tests/ test suite and on how to adapt them to the new layout for src. To achieve these goals, this change: - Moves tests from tools/regression/bin/<tool>/ to bin/<tool>/tests/. - Renames the previous regress.sh files to legacy_test.sh. - Adds Makefiles to build and install the tests and all their supporting data files into /usr/tests/bin/. - Plugs the legacy_test test programs into the test suite using the new TAP backend for Kyua (appearing in 0.8) so that the code of the test programs does not have to change. - Registers the new directories in the BSD.test.dist mtree file. Reviewed by: freebsd-testing Approved by: rpaulo (mentor)
2013-12-10 23:09:17 -05:00
# $FreeBSD$
First pass to fix the 'tests' packages. Sponsored by: The FreeBSD Foundation
2016-02-02 17:26:49 -05:00
PACKAGE= tests
Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed after r298107 Summary of changes: - Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that namespacing is kept with FILES appropriately, and that this shouldn't need to be repeated if the namespace changes -- only the definition of PACKAGE needs to be changed - Allow PACKAGE to be overridden by callers instead of forcing it to always be `tests`. In the event we get to the point where things can be split up enough in the base system, it would make more sense to group the tests with the blocks they're a part of, e.g. byacc with byacc-tests, etc - Remove PACKAGE definitions where possible, i.e. where FILES wasn't used previously. - Remove unnecessary TESTSPACKAGE definitions; this has been elided into bsd.tests.mk - Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES; ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk. - Fix installation of files under data/ subdirectories in lib/libc/tests/hash and lib/libc/tests/net/getaddrinfo - Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup) Document the proposed changes in share/examples/tests/tests/... via examples so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of replacing FILES. share/mk/bsd.README didn't seem like the appropriate method of communicating that info. MFC after: never probably X-MFC with: r298107 PR: 209114 Relnotes: yes Tested with: buildworld, installworld, checkworld; buildworld, packageworld Sponsored by: EMC / Isilon Storage Division
2016-05-04 19:20:53 -04:00
Convert bin/sh/tests to ATF The new code uses a "test discovery mechanism" to determine what tests are available for execution The test shell can be specified via: kyua test -v test_suites.FreeBSD.bin.sh.test_shell=/path/to/test/sh Sponsored by: EMC / Isilon Storage Division Approved by: jmmv (mentor) Reviewed by: jilles (maintainer)
2014-08-13 00:14:50 -04:00
TESTSDIR= ${TESTSBASE}/bin/sh/${.CURDIR:T}
Migrate tools/regression/bin/ tests to the new layout. This change is a proof of concept on how to easily integrate existing tests from the tools/regression/ hierarchy into the /usr/tests/ test suite and on how to adapt them to the new layout for src. To achieve these goals, this change: - Moves tests from tools/regression/bin/<tool>/ to bin/<tool>/tests/. - Renames the previous regress.sh files to legacy_test.sh. - Adds Makefiles to build and install the tests and all their supporting data files into /usr/tests/bin/. - Plugs the legacy_test test programs into the test suite using the new TAP backend for Kyua (appearing in 0.8) so that the code of the test programs does not have to change. - Registers the new directories in the BSD.test.dist mtree file. Reviewed by: freebsd-testing Approved by: rpaulo (mentor)
2013-12-10 23:09:17 -05:00
Convert bin/sh/tests to ATF The new code uses a "test discovery mechanism" to determine what tests are available for execution The test shell can be specified via: kyua test -v test_suites.FreeBSD.bin.sh.test_shell=/path/to/test/sh Sponsored by: EMC / Isilon Storage Division Approved by: jmmv (mentor) Reviewed by: jilles (maintainer)
2014-08-13 00:14:50 -04:00
.PATH: ${.CURDIR:H}
ATF_TESTS_SH= functional_test
Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed after r298107 Summary of changes: - Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that namespacing is kept with FILES appropriately, and that this shouldn't need to be repeated if the namespace changes -- only the definition of PACKAGE needs to be changed - Allow PACKAGE to be overridden by callers instead of forcing it to always be `tests`. In the event we get to the point where things can be split up enough in the base system, it would make more sense to group the tests with the blocks they're a part of, e.g. byacc with byacc-tests, etc - Remove PACKAGE definitions where possible, i.e. where FILES wasn't used previously. - Remove unnecessary TESTSPACKAGE definitions; this has been elided into bsd.tests.mk - Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES; ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk. - Fix installation of files under data/ subdirectories in lib/libc/tests/hash and lib/libc/tests/net/getaddrinfo - Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup) Document the proposed changes in share/examples/tests/tests/... via examples so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of replacing FILES. share/mk/bsd.README didn't seem like the appropriate method of communicating that info. MFC after: never probably X-MFC with: r298107 PR: 209114 Relnotes: yes Tested with: buildworld, installworld, checkworld; buildworld, packageworld Sponsored by: EMC / Isilon Storage Division
2016-05-04 19:20:53 -04:00
${PACKAGE}FILES+= bg1.0
${PACKAGE}FILES+= bg2.0
${PACKAGE}FILES+= bg3.0
${PACKAGE}FILES+= bg4.0
${PACKAGE}FILES+= bg5.0
${PACKAGE}FILES+= bg6.0 bg6.0.stdout
${PACKAGE}FILES+= bg7.0
${PACKAGE}FILES+= bg8.0
${PACKAGE}FILES+= bg9.0
${PACKAGE}FILES+= bg10.0 bg10.0.stdout
sh/tests: Add tests for SIGINT in non-jobc background commands If job control is not enabled, background commands shall ignore SIGINT and SIGQUIT, and it shall be possible to override that ignore in the same shell. MFC after: 1 week
2020-06-14 15:41:24 -04:00
${PACKAGE}FILES+= bg11.0
${PACKAGE}FILES+= bg12.0
sh: Add test for exported but unset variables PR: 233545
2019-01-03 15:23:12 -05:00
${PACKAGE}FILES+= env1.0
Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed after r298107 Summary of changes: - Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that namespacing is kept with FILES appropriately, and that this shouldn't need to be repeated if the namespace changes -- only the definition of PACKAGE needs to be changed - Allow PACKAGE to be overridden by callers instead of forcing it to always be `tests`. In the event we get to the point where things can be split up enough in the base system, it would make more sense to group the tests with the blocks they're a part of, e.g. byacc with byacc-tests, etc - Remove PACKAGE definitions where possible, i.e. where FILES wasn't used previously. - Remove unnecessary TESTSPACKAGE definitions; this has been elided into bsd.tests.mk - Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES; ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk. - Fix installation of files under data/ subdirectories in lib/libc/tests/hash and lib/libc/tests/net/getaddrinfo - Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup) Document the proposed changes in share/examples/tests/tests/... via examples so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of replacing FILES. share/mk/bsd.README didn't seem like the appropriate method of communicating that info. MFC after: never probably X-MFC with: r298107 PR: 209114 Relnotes: yes Tested with: buildworld, installworld, checkworld; buildworld, packageworld Sponsored by: EMC / Isilon Storage Division
2016-05-04 19:20:53 -04:00
${PACKAGE}FILES+= fork1.0
${PACKAGE}FILES+= fork2.0
${PACKAGE}FILES+= fork3.0
${PACKAGE}FILES+= func1.0
${PACKAGE}FILES+= func2.0
${PACKAGE}FILES+= func3.0
${PACKAGE}FILES+= hash1.0
${PACKAGE}FILES+= int-cmd1.0
${PACKAGE}FILES+= killed1.0
${PACKAGE}FILES+= killed2.0
${PACKAGE}FILES+= not1.0
${PACKAGE}FILES+= not2.0
${PACKAGE}FILES+= path1.0
sh: Add set -o pipefail The pipefail option allows checking the exit status of all commands in a pipeline more easily, at a limited cost of complexity in sh itself. It works similarly to the option in bash, ksh93 and mksh. Like ksh93 and unlike bash and mksh, the state of the option is saved when a pipeline is started. Therefore, even in the case of commands like A | B & a later change of the option does not change the exit status, the same way (A | B) & works. Since SIGPIPE is not handled specially, more work in the script is required for a proper exit status for pipelines containing commands such as head that may terminate successfully without reading all input. This can be something like ( cmd1 r=$? if [ "$r" -gt 128 ] && [ "$(kill -l "$r")" = PIPE ]; then exit 0 else exit "$r" fi ) | head PR: 224270 Relnotes: yes
2019-02-24 16:05:13 -05:00
${PACKAGE}FILES+= pipefail1.0
${PACKAGE}FILES+= pipefail2.42
${PACKAGE}FILES+= pipefail3.42
${PACKAGE}FILES+= pipefail4.42
${PACKAGE}FILES+= pipefail5.42
${PACKAGE}FILES+= pipefail6.42
${PACKAGE}FILES+= pipefail7.0
Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed after r298107 Summary of changes: - Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that namespacing is kept with FILES appropriately, and that this shouldn't need to be repeated if the namespace changes -- only the definition of PACKAGE needs to be changed - Allow PACKAGE to be overridden by callers instead of forcing it to always be `tests`. In the event we get to the point where things can be split up enough in the base system, it would make more sense to group the tests with the blocks they're a part of, e.g. byacc with byacc-tests, etc - Remove PACKAGE definitions where possible, i.e. where FILES wasn't used previously. - Remove unnecessary TESTSPACKAGE definitions; this has been elided into bsd.tests.mk - Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES; ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk. - Fix installation of files under data/ subdirectories in lib/libc/tests/hash and lib/libc/tests/net/getaddrinfo - Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup) Document the proposed changes in share/examples/tests/tests/... via examples so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of replacing FILES. share/mk/bsd.README didn't seem like the appropriate method of communicating that info. MFC after: never probably X-MFC with: r298107 PR: 209114 Relnotes: yes Tested with: buildworld, installworld, checkworld; buildworld, packageworld Sponsored by: EMC / Isilon Storage Division
2016-05-04 19:20:53 -04:00
${PACKAGE}FILES+= redir1.0
${PACKAGE}FILES+= redir2.0
${PACKAGE}FILES+= redir3.0
${PACKAGE}FILES+= redir4.0
${PACKAGE}FILES+= redir5.0
${PACKAGE}FILES+= redir6.0
${PACKAGE}FILES+= redir7.0
sh: Add simple test for 'set -C' (noclobber). To ensure fast test runs, race conditions are not tested.
2016-11-02 18:33:37 -04:00
${PACKAGE}FILES+= set-C1.0
Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed after r298107 Summary of changes: - Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that namespacing is kept with FILES appropriately, and that this shouldn't need to be repeated if the namespace changes -- only the definition of PACKAGE needs to be changed - Allow PACKAGE to be overridden by callers instead of forcing it to always be `tests`. In the event we get to the point where things can be split up enough in the base system, it would make more sense to group the tests with the blocks they're a part of, e.g. byacc with byacc-tests, etc - Remove PACKAGE definitions where possible, i.e. where FILES wasn't used previously. - Remove unnecessary TESTSPACKAGE definitions; this has been elided into bsd.tests.mk - Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES; ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk. - Fix installation of files under data/ subdirectories in lib/libc/tests/hash and lib/libc/tests/net/getaddrinfo - Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup) Document the proposed changes in share/examples/tests/tests/... via examples so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of replacing FILES. share/mk/bsd.README didn't seem like the appropriate method of communicating that info. MFC after: never probably X-MFC with: r298107 PR: 209114 Relnotes: yes Tested with: buildworld, installworld, checkworld; buildworld, packageworld Sponsored by: EMC / Isilon Storage Division
2016-05-04 19:20:53 -04:00
${PACKAGE}FILES+= set-n1.0
${PACKAGE}FILES+= set-n2.0
${PACKAGE}FILES+= set-n3.0
${PACKAGE}FILES+= set-n4.0
${PACKAGE}FILES+= set-x1.0
${PACKAGE}FILES+= set-x2.0
${PACKAGE}FILES+= set-x3.0
${PACKAGE}FILES+= set-x4.0
${PACKAGE}FILES+= shellproc1.0
sh: Test that executing various binary files is rejected If executing a file fails with an [ENOEXEC] error, the shell executes the file as a shell script, except that this execution may instead result in an error message if the file is binary. Per a recent Austin Group interpretation, we will need to change this to allow a concatenation of a shell script and a binary payload. See Austin Group bugs #1226 and #1250. MFC after: 1 week
2019-12-30 16:32:55 -05:00
${PACKAGE}FILES+= shellproc2.0
${PACKAGE}FILES+= shellproc3.0
${PACKAGE}FILES+= shellproc4.0
${PACKAGE}FILES+= shellproc5.0
sh: Allow more scripts without #! Austin Group bugs #1226 and #1250 changed the requirements for shell scripts without #! (POSIX does not specify #!; this is about the shell execution when execve(2) returns an [ENOEXEC] error). POSIX says we shall allow execution if the initial part intended to be parsed by the shell consists of characters and does not contain the NUL character. This allows concatenating a shell script (ending with exec or exit) and a binary payload. In order to reject common binary files such as PNG images, check that there is a lowercase letter or expansion before the last newline before the NUL character, in addition to the check for the newline character suggested by POSIX.
2020-05-30 12:00:49 -04:00
${PACKAGE}FILES+= shellproc6.0
Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed after r298107 Summary of changes: - Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that namespacing is kept with FILES appropriately, and that this shouldn't need to be repeated if the namespace changes -- only the definition of PACKAGE needs to be changed - Allow PACKAGE to be overridden by callers instead of forcing it to always be `tests`. In the event we get to the point where things can be split up enough in the base system, it would make more sense to group the tests with the blocks they're a part of, e.g. byacc with byacc-tests, etc - Remove PACKAGE definitions where possible, i.e. where FILES wasn't used previously. - Remove unnecessary TESTSPACKAGE definitions; this has been elided into bsd.tests.mk - Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES; ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk. - Fix installation of files under data/ subdirectories in lib/libc/tests/hash and lib/libc/tests/net/getaddrinfo - Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup) Document the proposed changes in share/examples/tests/tests/... via examples so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of replacing FILES. share/mk/bsd.README didn't seem like the appropriate method of communicating that info. MFC after: never probably X-MFC with: r298107 PR: 209114 Relnotes: yes Tested with: buildworld, installworld, checkworld; buildworld, packageworld Sponsored by: EMC / Isilon Storage Division
2016-05-04 19:20:53 -04:00
${PACKAGE}FILES+= subshell1.0 subshell1.0.stdout
${PACKAGE}FILES+= subshell2.0
${PACKAGE}FILES+= subshell3.0
${PACKAGE}FILES+= subshell4.0
${PACKAGE}FILES+= unknown1.0
sh: Fix double INTON with vfork The shell maintains a count of the number of times SIGINT processing has been disabled via INTOFF, so SIGINT processing resumes when all disables have enabled again (INTON). If an error occurs in a vfork() child, the processing of the error enables SIGINT processing again, and the INTON in vforkexecshell() causes the count to become negative. As a result, a later INTOFF may not actually disable SIGINT processing. This might cause memory corruption if a SIGINT arrives at an inopportune time. As of r360452, it causes the shell to abort when it would unsafely allocate or free memory in certain ways. Note that various places such as errors in non-special builtins unconditionally reset the count to 0, so the problem might still not always be visible. PR: 246497 Reported by: jbeich MFC after: 2 weeks
2020-05-16 12:29:23 -04:00
${PACKAGE}FILES+= unknown2.0
Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed after r298107 Summary of changes: - Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that namespacing is kept with FILES appropriately, and that this shouldn't need to be repeated if the namespace changes -- only the definition of PACKAGE needs to be changed - Allow PACKAGE to be overridden by callers instead of forcing it to always be `tests`. In the event we get to the point where things can be split up enough in the base system, it would make more sense to group the tests with the blocks they're a part of, e.g. byacc with byacc-tests, etc - Remove PACKAGE definitions where possible, i.e. where FILES wasn't used previously. - Remove unnecessary TESTSPACKAGE definitions; this has been elided into bsd.tests.mk - Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES; ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk. - Fix installation of files under data/ subdirectories in lib/libc/tests/hash and lib/libc/tests/net/getaddrinfo - Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup) Document the proposed changes in share/examples/tests/tests/... via examples so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of replacing FILES. share/mk/bsd.README didn't seem like the appropriate method of communicating that info. MFC after: never probably X-MFC with: r298107 PR: 209114 Relnotes: yes Tested with: buildworld, installworld, checkworld; buildworld, packageworld Sponsored by: EMC / Isilon Storage Division
2016-05-04 19:20:53 -04:00
${PACKAGE}FILES+= var-assign1.0
Migrate tools/regression/bin/ tests to the new layout. This change is a proof of concept on how to easily integrate existing tests from the tools/regression/ hierarchy into the /usr/tests/ test suite and on how to adapt them to the new layout for src. To achieve these goals, this change: - Moves tests from tools/regression/bin/<tool>/ to bin/<tool>/tests/. - Renames the previous regress.sh files to legacy_test.sh. - Adds Makefiles to build and install the tests and all their supporting data files into /usr/tests/bin/. - Plugs the legacy_test test programs into the test suite using the new TAP backend for Kyua (appearing in 0.8) so that the code of the test programs does not have to change. - Registers the new directories in the BSD.test.dist mtree file. Reviewed by: freebsd-testing Approved by: rpaulo (mentor)
2013-12-10 23:09:17 -05:00
.include <bsd.test.mk>
Reference in a new issue Copy permalink