Commit graph

488 commits

Author SHA1 Message Date
Brad Warren
a9edc0ae64 Merge branch 'master' into buildkit 2023-04-07 08:18:07 -07:00
humanoid2050
537ef57b0a removed subshell invocation of arch parsing method 2023-04-06 18:59:56 -04:00
humanoid2050
9b20628cde
Update tools/docker/lib/common
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-04-06 18:35:07 -04:00
humanoid2050
4582b7f726
Update tools/docker/lib/common
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-04-06 18:33:37 -04:00
humanoid2050
6f611a5869
Update tools/docker/lib/common
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-04-06 18:33:15 -04:00
Brad Warren
a78073812c
Always "pipstrap" when running pip_install.py (#9658)
Fixes https://github.com/certbot/certbot/issues/7921.

In all cases when we run `pip_install.py`, we first run `pipstrap.py`. This PR combines these two steps for convenience and to make always doing that less error prone. This will also help me with some of the `tox.ini` refactoring I'm planning to do.

I ran the full test suite on everything and tested the release script changes locally.

This change shouldn't have any effect on cryptography's setup because they install `certbot[test]` which depends on pip, setuptools, and wheel.

* always pipstrap

* use pip_install.py during releases
2023-04-05 16:43:26 -07:00
humanoid2050
960e678fcf improved quoting in shell scripts 2023-04-05 18:31:33 -04:00
humanoid2050
f01bc93f2c
Update tools/docker/lib/common
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-04-05 18:22:53 -04:00
humanoid2050
ec7851ca97 remove duplicate calls to multiarch install 2023-04-04 19:04:38 -04:00
humanoid2050
9de082a5f6 removed unnecessary testing script and fixed function name 2023-04-03 21:01:44 -04:00
humanoid2050
4d7ed3f692 refactor docker script arg parsing and fix merge bugs 2023-03-31 15:29:22 -04:00
Brad Warren
097af18417
remove readlink (#9649) 2023-03-31 10:02:12 +11:00
humanoid2050
a8c85ab0f8
Update tools/docker/deploy_manifests.sh
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-03-30 18:58:51 -04:00
humanoid2050
954eb02e2e
Update tools/docker/deploy_images.sh
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-03-30 18:58:40 -04:00
humanoid2050
e43b4db3bc
Update tools/docker/lib/common
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-03-30 18:53:22 -04:00
humanoid2050
c0e33b9dfa
Update tools/docker/build.sh
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-03-29 20:12:00 -04:00
humanoid2050
684f78c86c
Update tools/docker/build.sh
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-03-29 20:09:42 -04:00
humanoid2050
11d765ba4b
Update tools/docker/test.sh
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-03-29 19:51:42 -04:00
humanoid2050
c7adb00c9a
Update tools/docker/deploy_images.sh
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-03-29 19:34:20 -04:00
humanoid2050
cebe25e26f
Update tools/docker/build.sh
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-03-29 19:30:30 -04:00
humanoid2050
378926feac
Update tools/docker/build.sh
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-03-29 19:30:06 -04:00
humanoid2050
811e51dde8
Update tools/docker/README.md
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-03-29 19:28:30 -04:00
Brad Warren
208ef4eb94
remove CERTBOT_NO_PIN (#9634)
Adrien and I added this is in https://github.com/certbot/certbot/pull/6590 in response to https://github.com/certbot/certbot/issues/6582 which I wrote. I now personally think these tests are way more trouble than they're worth.

In almost all cases, the versions pinned in `tools/requirements.txt` are used. The two exceptions to this that come to mind are users using OS packages and pip. In the former, the version of our dependencies is picked by the OS and do not change much on most systems. As for pip, [we only "support it on a best effort basis"](https://eff-certbot.readthedocs.io/en/stable/install.html#alternative-2-pip).

Even for pip users, I'm not convinced this buys us much other than frequent test failures. We have our tests configured to error on all Python warnings and [we regularly update `tools/requirements.txt`](https://github.com/certbot/certbot/commits/master/tools/requirements.txt). Due to that, assuming our dependencies follow normal conventions, we should have a chance to fix things in response to planned API changes long before they make their way to our users. I do not think it is necessary for our tests to break immediately after an API is deprecated.

I think almost all other failures due to these tests are caused by upstream bugs. In my experience, almost all of them will sort themselves out pretty quickly. I think that responding to those that are not or planned API changes we somehow missed can be addressed when `tools/requirements.txt` is updated or when someone opens an issue. I personally don't think blocking releases or causing our nightly tests to fail is at all worth it here. I think removing this frequent cause of test failures makes things just a little bit easier for Certbot devs without costing us much of anything.
2023-03-27 17:01:27 -07:00
humanoid2050
678b9b485d revert buildx bake to bash for-loops 2023-03-25 15:03:08 -04:00
humanoid2050
adf227fc48 purge cache for build predictability 2023-03-19 15:21:51 -04:00
humanoid2050
c10192b6d8 leverage docker buildkit bake to speed up build times 2023-03-18 23:25:12 -04:00
humanoid2050
accee97476 bugfixes 2023-03-06 18:39:25 -05:00
humanoid2050
d78cde1894 Fix docker cache patterns 2023-03-06 17:28:10 -05:00
humanoid2050
e5f13f8ab0 testing and debuggin 2023-03-05 10:56:14 -05:00
humanoid2050
971ed2542a Move to multistage Dockerfile 2023-03-04 11:15:54 -05:00
humanoid2050
644328170c Remove qemu and switch to build via buildkit 2023-03-04 11:14:55 -05:00
humanoid2050
a42cffc351
generate multiarch images for non-architecture tags (#9586)
* generate multiarch images for non-architecture tags

* lock docker build to legacy docker buider, and bugfix

* rename deploy.sh to deploy_by_arch.sh

* Update documentation related to multiarch Docker

* Consistent IFS value with respect to other scripts

Co-authored-by: humanoid2050 <humanoid2050@monolith>
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-02-22 17:53:45 -08:00
humanoid2050
ac573e758e formatting fixes 2023-02-21 21:27:45 -05:00
humanoid2050
55f67ba74a
Consistent IFS value with respect to other scripts
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-02-21 21:26:20 -05:00
humanoid2050
d7c647f2c4
Merge branch 'master' into master 2023-02-18 13:37:59 -05:00
humanoid2050
cd4f7fcccd Update documentation related to multiarch Docker 2023-02-18 13:36:15 -05:00
humanoid2050
02d4911e6e rename deploy.sh to deploy_by_arch.sh 2023-02-18 13:22:14 -05:00
humanoid2050
c17bb3c032 lock docker build to legacy docker buider, and bugfix 2023-02-18 11:28:59 -05:00
humanoid2050
97ad18eddd
Apply suggestions from code review
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2023-02-18 11:24:35 -05:00
Brad Warren
2a7eeef176
Polish snap_build.py (#9584)
I wanted to try to make our tooling's messaging about it a little clearer.

While fixing my typo/bad English, we happened to hit a "Chroot problem" failure! See the logs for the CI first attempt at https://dev.azure.com/certbot/certbot/_build/results?buildId=6416&view=results.

Looking at these logs, I noticed three things:

1. This message I added is sometimes printed many times because we're still processing output from snapcraft. See https://dev.azure.com/certbot/certbot/_build/results?buildId=6416&view=logs&j=f44d40a4-7318-5ffe-762c-ae4557889284&s=1dfbc15b-7d0f-52a9-b1da-b17592bf94f8&t=07786725-57f8-5198-4d13-ea77f640bd5c&l=565.
2. snapcraft is complaining that we should be using --build-for now instead of --build-on. See https://dev.azure.com/certbot/certbot/_build/results?buildId=6416&view=logs&j=f44d40a4-7318-5ffe-762c-ae4557889284&s=1dfbc15b-7d0f-52a9-b1da-b17592bf94f8&t=07786725-57f8-5198-4d13-ea77f640bd5c&l=472.
3. Us canceling the Certbot build due to a "Chroot problem" happened 3 times in 3 seconds which seems very unlikely. See https://dev.azure.com/certbot/certbot/_build/results?buildId=6416&view=logs&j=f44d40a4-7318-5ffe-762c-ae4557889284&s=1dfbc15b-7d0f-52a9-b1da-b17592bf94f8&t=07786725-57f8-5198-4d13-ea77f640bd5c&l=587. I looked at the builds on launchpad and I only saw one Certbot build. I think what's happening is this code is causing the old build state to be reported so we error immediately.
I fixed all of these things in my follow up commits.

* polish chroot problem messaging

* only execute branch once
2023-02-16 11:17:47 -08:00
humanoid2050
43e89aee00 generate multiarch images for non-architecture tags 2023-02-13 21:09:18 -05:00
Brad Warren
1bb09da270
Update and run isort (#9573)
I want to use isort as part of https://github.com/certbot/certbot/issues/9572 because I want to do it programmatically, however, I felt like the config needed to be tweaked a bit due to it not understanding what is and is not our own code.

This PR updates the isort config so it recognizes our own modules and runs `isort .` from the root of the repo to update everything.

* update isort config

* run "isort ."
2023-02-10 10:51:20 -08:00
Brad Warren
7d4535a836 tweak cloudxns condition 2023-02-09 19:35:37 -08:00
Brad Warren
f91d3ca828 remove 1.32.x deps 2023-02-09 18:42:41 -08:00
Brad Warren
3512d15dff Remove most progressive release tooling 2023-02-09 18:41:16 -08:00
alexzorin
99184daff6
repin cryptography for openssl security update (#9565)
* repin cryptography for openssl security update

https://www.openssl.org/news/secadv/20230207.txt
https://cryptography.io/en/latest/changelog/#v39-0-1

* fix type hints

* remove outdated comments
2023-02-08 11:17:44 -08:00
Brad Warren
00f8d82808
double progressive percentage (#9557) 2023-02-01 07:05:01 +11:00
Brad Warren
b1f22aa8a2
Add progressive release tooling (#9532)
This is based on what I wrote at https://opensource.eff.org/eff-open-source/pl/k1b4pcxnifyj9m7o4wdq7cka8h.
2023-01-11 12:27:38 -08:00
Brad Warren
666e12b25d
upgrade shellingham (#9529) 2023-01-05 19:30:47 +11:00
Brad Warren
8155d60e9a
remove setuptools pin (#9520) 2022-12-21 10:59:41 +11:00