certbot/tools
Adrien Ferrand d434b92945
Build the DNS plugins snaps (#8129)
Fixes #8041

This PR makes Azure Pipeline build the DNS plugins snaps for the 3 architectures during the CI.

It leverages the existing logic for building the Certbot snap in order to deploy a QEMU environment with Docker, and leverages the local PyPI index to speed up the build when installing `cffi` and `cryptography`.

All DNS plugins snaps are constructed in one unique docker container, in order to save the time required to install the system dependencies upon first start of `snapcraft`, and so speed up significantly the build.

Finally, all `amd64` DNS plugins snaps are built within 6 minutes. For `arm64` and `armhf`, it is around 40 mins: this is quite fast in fact, considering that 14 DNS plugins snaps are built.

However, this is still an extremely heavy task to make the full 3 architectures builds, even for Azure Pipelines and its 10 parallel jobs capability. That is why I make the `arm64` and `armhf` builds be skipped for the `full-test-suite`, and let them run only for `nightly` and `release`. This means however that these builds will not be done for the release branches. If this is a problem, I can put a more elaborate suspend condition to triggers the builds in this case.

All snaps are stored in the pipeline artifacts storage, making them available for publication during a `release` pipeline.

The PR is set as Draft for now, because I use temporarily `pr_test-suite` to validate the packaging jobs when commits are pushed. Once the PR is ready, I will revert it back to the normal configuration (run the standard tests).

* Configure a script to build DNS snaps

* Focus on packaging

* Trigger all architectures

* Add extra index

* Prepare conditional suspend

* Set final suspend logic

* Set final suspend value

* Loop for publication

* Use python3

* Clean before build

* Add a test

* Add test job in Azure

* Preserve env

* Apply normal config for pipelines

* Skip QEMU jobs only for test branches

* Makes snap run tests depends also on the Certbot snap build

* Update .azure-pipelines/templates/jobs/packaging-jobs.yml

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>

* Update .azure-pipelines/templates/stages/deploy-stage.yml

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>

* More accurate way to get the plugin snap name

* Integrate DNS snap tests into certbot-ci

* Fixes

* Update certbot-ci/snap_integration_tests/conftest.py

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>

* Update certbot-ci/snap_integration_tests/conftest.py

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>

* Clean an _init_.py file

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2020-07-09 11:33:25 -07:00
..
snap Build the DNS plugins snaps (#8129) 2020-07-09 11:33:25 -07:00
_changelog_top.txt Remove list of modified packages from changelog. (#7146) 2019-06-11 14:02:54 -07:00
_release.sh Use Python 3 in the release script. (#7918) 2020-04-20 14:44:53 -07:00
_venv_common.py Fix TLS-ALPN tests with newer versions of OpenSSL (#8026) 2020-06-01 15:18:38 -07:00
deactivate.py Minor bugfixes (#7891) 2020-04-13 10:41:39 -07:00
dev_constraints.txt [Apache v2] Load apacheconfig tree and gate related tests (#7710) 2020-03-23 17:05:22 -07:00
eff-pubkey.pem Add pubkey to tree 2016-03-01 11:21:51 -08:00
extract_changelog.py Reorganize imports (#7616) 2019-12-09 15:50:20 -05:00
generate_dnsplugins_snapcraft.sh Make DNS plugin snaps use core20 (#8106) 2020-06-23 09:31:08 -07:00
half-sign.c Use SHA256 openssl signatures 2016-01-11 14:22:51 -08:00
install_and_test.py Remove letshelp-certbot (#7761) 2020-02-14 17:19:19 -08:00
merge_requirements.py Address review comments 2019-03-04 15:52:38 +02:00
offline-sigrequest.sh Don't require festival during signing. (#6079) 2018-06-07 11:43:45 +03:00
oldest_constraints.txt Migrate the CI pipeline from Travis to Azure Pipeline (#8098) 2020-07-02 15:01:21 -07:00
pip_install.py Reorganize imports (#7616) 2019-12-09 15:50:20 -05:00
pip_install_editable.py Add comment about pyca's use of tools script (#8044) 2020-06-08 12:14:02 -07:00
readlink.py Reorganize imports (#7616) 2019-12-09 15:50:20 -05:00
release.sh Release script improvements (#6337) 2018-09-05 14:10:05 -07:00
run_oldest_tests.sh Migrate the CI pipeline from Travis to Azure Pipeline (#8098) 2020-07-02 15:01:21 -07:00
simple_http_server.py Some improvements (#8132) 2020-07-06 16:04:59 -07:00
sphinx-quickstart.sh Don't display todo comments in docs (#7753) 2020-02-06 15:39:47 -08:00
strip_hashes.py Address review comments 2019-03-04 15:52:38 +02:00
venv.py Reorganize imports (#7616) 2019-12-09 15:50:20 -05:00
venv3.py Refactor certbot/ and certbot/tests/ to use the same structure as the other packages (#7544) 2019-11-25 14:28:05 -08:00