Tarball uploading steps are to be included in the pipeline.
They are manual jobs that:
1. uploads a release tarball prepared by the "sign" job.
2. publish a previously staged release to a destination.
3. publishes a previously staged release to a well-known URL.
A template has been included since all three follow the same pattern.
Besides, rules commonly used by tag pipeline jobs.
Merge branch 'andoni/ci-release-process' into 'main'
See merge request isc-projects/bind9!11158
Define and use more YAML anchors for triggering rules commonly used by
tag pipeline jobs. This builds on top of the work done in commit
675d9c7425, improving readability and
reuse.
Add a new SSH-confirmed GitLab CI job that publishes a previously staged
release to a well-known URL. The details of what specifically this
entails are controlled by the staging environment.
Add a new SSH-confirmed GitLab CI job that publishes a previously staged
release to a destination that is not a well-known URL. The details of
what specifically this entails are controlled by the staging
environment.
Add a YAML template for jobs that require an SSH connection to a
dedicated, locked-down runner for signing off on sensitive operations
(e.g. signing, publishing).
These jobs all follow a similar scheme:
1. Runner prepares the necessary files in a well-known location (/tmp).
2. Runner generates a shell script to be run by an authorized user.
3. Runner sleeps while waiting for a signal that the script was run.
4. Authorized user logs in to the runner over SSH and runs the script.
5. Runner collects the relevant files and logs as job artifacts.
One additional complication is that each of the above steps needs to be
carried out under the assumption that GitLab Runner is running under a
different user account than the one used for logging in over SSH,
necessitating careful file permission handling.
Having a YAML template for jobs that need to follow the above scheme
significantly improves readability and reuse as each job only needs to
define (via the "variables" YAML key):
- SSH_SCRIPT_RUNNER_PRE: the code the runner should execute before an
authorized user logs in over SSH (typically: setting up files in a
well-known location),
- SSH_SCRIPT_CLIENT: contents of the shell script to be run by an
authorized user,
- SSH_SCRIPT_RUNNER_POST: the code the runner should execute after an
authorized user runs the script over SSH (typically: artifact
collection and cleanup).
Include the Git tag in the name of the release directory rather than
just the version number. Revise the script for the "release" job
accordingly. This enables using the $CI_COMMIT_TAG variable in job
scripts without the need to resort to string manipulation to strip the
leading "v", improving readability. The only place where string
manipulation is applied to the Git tag is now the "release" job itself,
to verify that the Git tag matches the version number embedded in the
source tarball name.
The variables example2com, example3com, and example4com where not
being initalised on all paths leading to Coverity issues ID 637690,
ID 637691 and ID 637692 being raised. In addition the tests to free
them were wrong as they depended on the unintialised variables. This
has been fixed.
Closes#5598
Merge branch '5598-properly-determine-if-name-is-dynamic' into 'main'
See merge request isc-projects/bind9!11155
The variables example2com, example3com, and example4com where not
being initalised on all paths leading to Coverity issues ID 637690,
ID 637691 and ID 637692 being raised. In addition the tests to free
them were wrong as they depended on the unintialised variables. This
has been fixed.
Passing the JSON_C_TO_STRING_PRETTY flag to
json_object_to_json_string_ext() makes the latter produce prettified
JSON output. This results in a huge amount of redundant whitespace
being inserted into each HTTP response (whitespace amounts to about 40%
of the entire JSON payload).
The bandwidth cost can be amortized by enabling HTTP compression on the
client side ("Accept-Encoding: deflate"), but that does not affect the
size of data at rest.
Use the JSON_C_TO_STRING_PLAIN flag instead of JSON_C_TO_STRING_PRETTY
to minimize the size of JSON responses sent via the statistics channel.
External tools should be used for prettifying JSON data.
Closes#3304
Merge branch '3304-stop-prettifying-json-statistics' into 'main'
See merge request isc-projects/bind9!10786
Passing the JSON_C_TO_STRING_PRETTY flag to
json_object_to_json_string_ext() makes the latter produce prettified
JSON output. This results in a huge amount of redundant whitespace
being inserted into each HTTP response (whitespace amounts to about 40%
of the entire JSON payload).
The bandwidth cost can be amortized by enabling HTTP compression on the
client side ("Accept-Encoding: deflate"), but that does not affect the
size of data at rest.
Use the JSON_C_TO_STRING_PLAIN flag instead of JSON_C_TO_STRING_PRETTY
to minimize the size of JSON responses sent via the statistics channel.
External tools should be used for prettifying JSON data.
Processing JSON and XML using `grep` and `sed` is error-prone, overly
lax in some ways, overly strict in others, and neither accurate nor
expressive. Use `jq` and `xmllint` with XPath expressions to make
things right in system tests.
See #3304
Merge branch '3304-properly-process-json-and-xml-in-tests' into 'main'
See merge request isc-projects/bind9!10942
The traffic-json.pl and traffic-xml.pl scripts that were used in the
"statschannel" system test in the past became dead code when commit
1202fd912a rewrote parts of that test to
Python. Remove those scripts.
There are no longer any HTML files in the BIND 9 source repository.
Remove the xmllint-html.sh script that was used in the past to check
those for errors.
Inspecting XML data using sed and grep is error-prone, overly lax in
some ways, overly strict in others, and neither accurate nor expressive.
Use xmllint and XPath expressions for inspecting XML data in the
"statistics", "statschannel", and "synthfromdnssec" system tests to
address these deficiencies.
Detect whether and where the xmllint utility is available at build time,
so that it can be used in system tests. If the tool is not found,
specific checks employing it will be skipped.
Inspecting JSON data using grep is error-prone, overly lax in some ways,
overly strict in others, and neither accurate nor expressive. Use jq
for inspecting JSON data in the "statschannel" and "synthfromdnssec"
system tests to address these deficiencies.
Detect whether and where the jq utility is available at build time, so
that it can be used in system tests. If the tool is not found, specific
checks employing it will be skipped.
When the arc4random_uniform() is called on NetBSD with upper_bound that
makes no sense statistically (0 or 1), the call crashes the calling
program. Fix this by returning 0 when upper bound is < 2 as does Linux,
FreeBSD and NetBSD. (Hint: System CSPRNG should never crash.)
Closes#5596
Merge branch '5596-fix-isc_random_uniform-on-NetBSD' into 'main'
See merge request isc-projects/bind9!11147
When the arc4random_uniform() is called on NetBSD with upper_bound that
makes no sense statistically (0 or 1), the call crashes the calling
program. Fix this by returning 0 when upper bound is < 2 as does Linux,
FreeBSD and NetBSD. (Hint: System CSPRNG should never crash.)
Since we are using system routines for randomness, there's no point
in spending time and run the statistical suite for testing PRNG.
Merge branch 'ondrej/drop-random-unit-test' into 'main'
See merge request isc-projects/bind9!11137
The minimal required meson version is 1.3.0.
Where distribution repositories don't provide meson 1.3.0 or newer, meson from PyPI may be used instead.
Merge branch 'aydin/meson-bump-version' into 'main'
See merge request isc-projects/bind9!10997
After a couple releases, it appears that the minimum version can be
increased without much trouble.
Because meson only requires python 3.7 or greater, most supported
distributions can run a newer meson without having to deal with
additional dependencies or a new python version.
Previously fuzzing builds were broken due to some typos in the `meson.build`.
Merge branch 'aydin/fuzz-meson-fix' into 'main'
See merge request isc-projects/bind9!11052
The fuzzing binary builds have been fixed and now is run as a part of
the test suite for sanity checks.
The `oss-fuzz` backend has been made functional with the addition of the
`oss-fuzz-args` option. It allows the setup script to pass the
LIB_FUZZING_ENGINE environment variable to meson.
When refactoring the BIND 9.21 build system to meson, libcap was
unintentionally made optional on Linux.
Closes#5590
Merge branch '5590-make-libcap-mandatory-on-linux-again' into 'main'
See merge request isc-projects/bind9!11136
After a merge into the public project, trigger automatic rebase in the respective private branch of the private project.
Merge branch 'pspacek/auto-rebase-sub-branches' into 'main'
See merge request isc-projects/bind9!11012
We do not want to re-release tags we create in the private project
anyway. Moreover pushing tags back into the public project after release
caused to Gitlab to create tag pipelines which never finished, which was
only confusing thing.
`cfg_obj_t` objects no longer depend on the `cfg_parser_t` life-cycle; they can now persist until the last reference is detached. The `file` field, which was previously a pointer to memory allocated in the parser, is now a pointer to a subsidiary `cfg_obj_t` of type string. The API calls for creating and detaching these objects have been simplified accordingly.
Since `cfg_obj_t` is now long-lived, a zone can hold a reference to its own configuration data, making it possible to use `rndc showzone` even if `allow-new-zones` is disabled.
Several API calls related to the parser have been removed or hidden. The `cfg_parse_file()` and `cfg_parse_buffer()` functions now internally create and destroy their own parsers, eliminating the need for the caller to do so.
Most of these changes are intended to simplify dumping of running configuration data in a future commit.
Merge branch 'colin/hide-cfg-parser' into 'main'
See merge request isc-projects/bind9!11132
change_directory() now lives in libisccfg. when it was moved,
the logging behavior changed: previously it had been logged
by named only, in the general logging category, and without the
named.conf filename and line number. it was not logged by
named-checkconf. this behavior has now been restored.
Since the zone now has a reference to their respective configuration
tree, `rndc showzone` can be used for any zones (including those
defined in namedconf), without `allow-new-zones` being enabled.
Add a test for this.
The test is part of the addzone suite because showzone used to be
related to addzone, but this could be moved elsewhere in the future
if more specific tests are needed for showzone.
Since the `file` property of cfg_obj_t can now be null (instead of
"none"), cfg_obj_t would take a fallback flow where the line was not
logged. This fixes it.
Also, add the log line when parser_complain is called and `file` is null
(which might happend when parsing buffer only) to also include the line
number.
- several functions that can no longer fail have been changed to
type void, and unnecessary 'cleanup' sections were removed
- renamed cfg_create_obj() to cfg_obj_create(), and cfg_create_tuple()
to cfg_tuple_create(), to match typical nomenclature.
- fixed a memory leak bug, in which an element could be removed
from a list in delete_zoneconf() without being freed. this has
been addressed by adding a cfg_list_unlink() function.
list elements are now allocated based on the list they will
be stored in, using the same mctx.
- the cfg_parser_create() and cfg_parser_destroy() calls are no
longer used outside parser.c, so they are now static functions
- cfg_parser_attach(), cfg_parser_reset(), and cfg_parser_setflags()
are no longer used at all, and have been removed.
- cfg_parser_mapadd() has been renamed for clarity to cfg_map_add().
Instead of (1) allocating a parser, (2) parsing a file/buffer then (3)
freeing the parser, the parser is now internally created/destroyed from
within the `cfg_parse_*` functions. This simplifies a lot the use cases,
especially around the error cases where the parser needs to be freed in
a cleanup goto.
The only trick was the parser callback mechanism, which would previously
have been set up between steps 1 and 2. Since it's never been used for
any purpose other than the "directory" option, the chdir call has now
been moved inside the parser and the generic callback mechanism has been
removed, replacing CFG_CLAUSEFLAG_CALLBACK with CFG_CLAUSEFLAG_CHDIR.
when configuring a zone, we can now save the zone's configuration
object in the zone itself by calling dns_zone_setcfg(). this can
then be used by "rndc showzone" to print the zone's configuration,
which is simpler than searching for it using the new-zones
configuration, and allows it to work even if "allow-new-zones"
is disabled.
Remove all global cfg_parser objects as well as shared parsers between
views to dynamically add zones. Instead, parser are transirently created
whenever needed.
