Fixes https://github.com/certbot/certbot/issues/7921.
In all cases when we run `pip_install.py`, we first run `pipstrap.py`. This PR combines these two steps for convenience and to make always doing that less error prone. This will also help me with some of the `tox.ini` refactoring I'm planning to do.
I ran the full test suite on everything and tested the release script changes locally.
This change shouldn't have any effect on cryptography's setup because they install `certbot[test]` which depends on pip, setuptools, and wheel.
* always pipstrap
* use pip_install.py during releases
This is a first step towards implementing the plan I described at https://github.com/certbot/certbot/issues/7909#issuecomment-1448675456 which got a +1 from both Erica and Will. Similar changes for our other packages will be made in followup PRs to try and make this easier to review.
It may be helpful to look at https://github.com/certbot/certbot/pull/7600 when reviewing this PR where we did something similar in the past.
The value of `ignore-paths` in `.pylintrc` should work on Windows based on https://pylint.readthedocs.io/en/latest/user_guide/configuration/all-options.html#ignore-paths and the fact that on macOS/linux, changing path delimiters to `\` still causes these directories to be ignored.
I started testing this for mypy as well, but mypy doesn't current pass for us on Windows so I didn't bother and took this opportunity to remove it from the default environments in `tox.ini`. I'll update https://github.com/certbot/certbot/issues/7803 to mention that the value of `exclude` in `mypy.ini` may need to be tweaked if anyone works on that issue.
* make acme tests internal
* no mypy-win
Adrien and I added this is in https://github.com/certbot/certbot/pull/6590 in response to https://github.com/certbot/certbot/issues/6582 which I wrote. I now personally think these tests are way more trouble than they're worth.
In almost all cases, the versions pinned in `tools/requirements.txt` are used. The two exceptions to this that come to mind are users using OS packages and pip. In the former, the version of our dependencies is picked by the OS and do not change much on most systems. As for pip, [we only "support it on a best effort basis"](https://eff-certbot.readthedocs.io/en/stable/install.html#alternative-2-pip).
Even for pip users, I'm not convinced this buys us much other than frequent test failures. We have our tests configured to error on all Python warnings and [we regularly update `tools/requirements.txt`](https://github.com/certbot/certbot/commits/master/tools/requirements.txt). Due to that, assuming our dependencies follow normal conventions, we should have a chance to fix things in response to planned API changes long before they make their way to our users. I do not think it is necessary for our tests to break immediately after an API is deprecated.
I think almost all other failures due to these tests are caused by upstream bugs. In my experience, almost all of them will sort themselves out pretty quickly. I think that responding to those that are not or planned API changes we somehow missed can be addressed when `tools/requirements.txt` is updated or when someone opens an issue. I personally don't think blocking releases or causing our nightly tests to fail is at all worth it here. I think removing this frequent cause of test failures makes things just a little bit easier for Certbot devs without costing us much of anything.
* Add async interface for finalization to acme.client.ClientV2
Add `begin_order_finalization()`/`poll_finalization()` to
`acme.client.ClientV2`, which are directly analogous to
`answer_challenge()`/`poll_authorizations()`. This allows us to
finalize an order and then later poll for its completion as separate
steps.
* Address code review feedback
Rename `begin_order_finalization` -> `begin_finalization` and tweak
wording of changelog entry
Right now if you to_json() an `OrderResource` and later deserialize
it, the `AuthorizationResource` objects don't come back through the
round-trip (they just get de-jsonified as frozendicts and worse, they
can't even be passed to `AuthorizationResource.from_json` because
frozendicts aren't dicts). In addition, the `csr_pem` field gets
encoded as an array of integers, which definitely does not get
de-jsonified into what we want.
Fix these by adding an encoder to `authorizations` and encoder and
decoder to `csr_pem`.
`lock_test.py` is a weird, heavily customized, standalone testing relic that's giving me trouble because the name currently conflicts with `certbot/tests/lock_test.py`. Moving `certbot/tests` inside the Certbot package as discussed at https://github.com/certbot/certbot/issues/7909#issuecomment-1448675456 would avoid this, however, this is at least somewhat blocked on getting that test code passing lint and mypy checks again because we run those checks on the entirety of the Certbot package 🙃 Since `lock_test.py` could probably stand to be rewritten/refactored anyway, I took this approach.
What I did is I rewrote something largely equivalent to `lock_test.py` inside Certbot's unit tests. I chose not to do this in `certbot-ci` because its not necessary to have an ACME server available. We're no longer explicitly testing things with the nginx plugin here like we were in `lock_test.py`, however, we are checking that `prepare` is called on the plugin at the right time and I added comments about the importance of checking that we lock the directory during the call to `prepare` in the Apache and nginx test code.
As a bonus, this fixes https://github.com/certbot/certbot/issues/8121.
