3 commits

Author	SHA1	Message	Date
Adrien Ferrand	aeb7beb1b1	Fully type certbot-ci module (#9120) ... * Fully type certbot-ci module * Fix lint, focus lint * Add trailing comma * Remove unused private function * Type properly for future usages * Update certbot-ci/certbot_integration_tests/utils/acme_server.py Co-authored-by: alexzorin <alex@zor.io> * Cleanup files * Fix import * Fix mypy and lint Co-authored-by: alexzorin <alex@zor.io>	2021-11-30 08:24:39 +11:00
Adrien Ferrand	14dfbdbea5	Build snaps using the remote-build feature (#8153) ... Snapcraft has a feature name `remote-build`. It allows to compile snaps using the Canonical dedicated build architecture for several architectures. Compared to the QEMU-enabled Docker approach used currently, the remote build has several advantages: * the builds are done on the native architecture, making them basically faster than what can be achieved on QEMU * it avoids to depend on `adferrand/snapcraft` (which could be otherwise be fixed with the merge of https://github.com/snapcore/snapcraft/pull/3144, but this will not happen in the short term) * when everything is good, all snaps build can be run in parallel and then can be orchestrated by one single Azure Pipeline job, since the heavy tasks are done remotely. This PR makes the necessary ajustements to use the remote build feature instead of the QEMU-enabled docker approach. One complex task was to be able to compile the `certbot` snap on `arm64` and `armhf`. Indeed on these architectures the pre-compiled wheel for `cffi` is not available. So it needs to be compiled during the snap build. Sadly, the current version of the python plugin in snapcraft is limited by the fact that `wheels` is not installed in the virtual environment set up to build the python packages, and there is no easy way to change that except by overridding the whole build process. In the long term, I think I will open a PR on `snapcraft` Git repository to provide a consistent solution. But for the short term, I used the possibility to provide arguments to the `venv` module, to add the flag `--system-site-packages`. With it, the virtual environment can use the system site package, where `wheel` is available. The other significant additions are in `tools/snap/build_remote.py` script. If invoking the remote build on a local machine is quite straight-forward, it is another story on the CI because we need build auditability and resiliency during these non-interactive actions. In particular we should avoid as possible inconsistent results on the nightly pipeline and the release pipeline. So this script wraps the `snapcraft` call into a retry logic, and improves its logs in the context of parallel builds. For the minor modifications, it is mainly about ensuring that plugins can be built (some of them also need `cffi` for instance), and simplify the Azure Pipeline since all snaps are retrieved in one go. Please note that the `test-` branches still run only the `amd64` architecture. Indeed I noticed that builds on `arm64` and `armhf` are tending to be very slow to start (up to 40 min) while the `amd64` ones wait at max 10 mins, and usually 30 seconds only when the overall load on Canonical side is low. To work on `certbot/certbot` repository, one secured file needs to be added, because `snapcraft` needs to be authenticated against Launchpad with credentials allowing remote builds. To do so, from a local machine that have this capability, one can extract the existing file at `$HOME/.local/share/snapcraft/provider/launchpad/credentials`, and register it as a secured file in Azure Pipeline with the name `snapcraftRemoteBuildCredentials`. * Define scripts * Setup pipeline to use remote builds * Focus on packaging builds * Set credentials * Setup git * Launch all builds in parallel * Add dev dependencies to build cffi and cryptography * Convert to a python logic * Reorganize the pipeline * Handle the fact that snap builds may be taken from cache * Generate constraints * Exit code * Check existence * Try to handle better non zero exit code * Add --system-site-packages to get wheel in the venv * Add executable permissions * Troubleshoot * Dynamic display, take the maximum timeout for snap build job * Allow retries if the remote build does not start * Trigger only amd64 builds for test branches * Exit properly * Update snapcraft.yaml * Fix snap run * Set secured file name * Update .azure-pipelines/templates/jobs/packaging-jobs.yml Co-authored-by: Brad Warren <bmw@users.noreply.github.com> * Update .azure-pipelines/templates/jobs/packaging-jobs.yml Co-authored-by: Brad Warren <bmw@users.noreply.github.com> * Update .azure-pipelines/templates/jobs/packaging-jobs.yml Co-authored-by: Brad Warren <bmw@users.noreply.github.com> * Move order in deps * Reactivate all builds * Use Manager() as a context manager * Use Pool as a context manager * Some nice refactorings * Check snapcraft execution interruption with exit codes * Use f-string and format expressions * Start log * Consistent use of single/double quotes * Better loop to extract lines * Retry on build failures * Few optimizations Co-authored-by: Brad Warren <bmw@users.noreply.github.com>	2020-07-22 16:05:20 -07:00
Adrien Ferrand	d434b92945	Build the DNS plugins snaps (#8129) ... Fixes #8041 This PR makes Azure Pipeline build the DNS plugins snaps for the 3 architectures during the CI. It leverages the existing logic for building the Certbot snap in order to deploy a QEMU environment with Docker, and leverages the local PyPI index to speed up the build when installing `cffi` and `cryptography`. All DNS plugins snaps are constructed in one unique docker container, in order to save the time required to install the system dependencies upon first start of `snapcraft`, and so speed up significantly the build. Finally, all `amd64` DNS plugins snaps are built within 6 minutes. For `arm64` and `armhf`, it is around 40 mins: this is quite fast in fact, considering that 14 DNS plugins snaps are built. However, this is still an extremely heavy task to make the full 3 architectures builds, even for Azure Pipelines and its 10 parallel jobs capability. That is why I make the `arm64` and `armhf` builds be skipped for the `full-test-suite`, and let them run only for `nightly` and `release`. This means however that these builds will not be done for the release branches. If this is a problem, I can put a more elaborate suspend condition to triggers the builds in this case. All snaps are stored in the pipeline artifacts storage, making them available for publication during a `release` pipeline. The PR is set as Draft for now, because I use temporarily `pr_test-suite` to validate the packaging jobs when commits are pushed. Once the PR is ready, I will revert it back to the normal configuration (run the standard tests). * Configure a script to build DNS snaps * Focus on packaging * Trigger all architectures * Add extra index * Prepare conditional suspend * Set final suspend logic * Set final suspend value * Loop for publication * Use python3 * Clean before build * Add a test * Add test job in Azure * Preserve env * Apply normal config for pipelines * Skip QEMU jobs only for test branches * Makes snap run tests depends also on the Certbot snap build * Update .azure-pipelines/templates/jobs/packaging-jobs.yml Co-authored-by: Brad Warren <bmw@users.noreply.github.com> * Update .azure-pipelines/templates/stages/deploy-stage.yml Co-authored-by: Brad Warren <bmw@users.noreply.github.com> * More accurate way to get the plugin snap name * Integrate DNS snap tests into certbot-ci * Fixes * Update certbot-ci/snap_integration_tests/conftest.py Co-authored-by: Brad Warren <bmw@users.noreply.github.com> * Update certbot-ci/snap_integration_tests/conftest.py Co-authored-by: Brad Warren <bmw@users.noreply.github.com> * Clean an _init_.py file Co-authored-by: Brad Warren <bmw@users.noreply.github.com>	2020-07-09 11:33:25 -07:00