on main if you run tools/pinning/current/repin.sh and run our unit
tests, they will fail due to new deprecation warnings from pyparsing.
the cause of these warnings is described at
dc009668d8/docs/whats_new_in_3_0_0.rst (L613-L708)
this PR fixes these warnings and updates our minimum required pyparsing
version to 3.0 where the new naming convention is available. i ran our
full test suite on the first commit here and it passed
i don't think it's worth trying to keep compatibility with pyparsing<3
unless we get a request for us to do so which i really doubt we will
Fixes #10518.
`tools/pinning/current/repin.sh` is not run; only pytest version is
updated. This is because `pypinning` had a bunch of syntax changes that
seem simply but I believe should be in a separate PR, which I think
should be done after this to collect all repin changes.
As discussed further in #10518, these issues were caused by pytest's
internalization of pytest-subtest, which had several implementation
changes.
To fix these, we simply no longer use subtest in the failing tests. The
test in acme is now parametrized instead, and the tests in apache only
ever had a single parameter.
To use parametrization in the acme test, I converted `DNSTest` from
unittest to pytest style, which was pretty straightforward. The only
note there is that while it would be nice to make `ec_secp384r1_key` a
fixture, you [can't use fixtures in
parameters](https://github.com/pytest-dev/pytest/issues/349). You could
use requests, but that seemed less clear and messier, because then you'd
be checking the value of the parameter and only sometimes loading it.
Could also make it a global variable, but that didn't really seem
necessary, as it's only called twice. Happy to consider other options,
not strongly tied to this one, just seemed nicest to me.
this fixes the dependabot alerts those with access can see at
https://github.com/certbot/certbot/security/dependabot
i don't think those alerts are particularly relevant to us, but i think
it's good for us to update anyway
in https://github.com/canonical/snapcraft/pull/5720, snapcraft made a
change. `snapcraft status certbot` output changed from something like
this:
```
Track Arch Channel Version Revision Progress
latest amd64 stable 5.1.0 5057 -
candidate ↑ ↑ -
beta 5.2.1 5214 -
edge 5.2.0.dev0 5210 -
arm64 stable 5.1.0 5058 -
candidate ↑ ↑ -
beta 5.2.1 5215 -
edge 5.2.0.dev0 5211 -
armhf stable 5.1.0 5056 -
candidate ↑ ↑ -
beta 5.2.1 5213 -
edge 5.2.0.dev0 5212 -
```
to this:
```
Track Arch Channel Version Revision Progress
latest amd64 stable 5.1.0 5057 -
latest amd64 candidate ↑ ↑ -
latest amd64 beta 5.2.1 5214 -
latest amd64 edge 5.2.0.dev0 5210 -
latest arm64 stable 5.1.0 5058 -
latest arm64 candidate ↑ ↑ -
latest arm64 beta 5.2.1 5215 -
latest arm64 edge 5.2.0.dev0 5211 -
latest armhf stable 5.1.0 5056 -
latest armhf candidate ↑ ↑ -
latest armhf beta 5.2.1 5213 -
latest armhf edge 5.2.0.dev0 5212 -
```
when its output is captured like it is in finish_release.py in the lines
above the code i'm modifying here
not matching on the beginning of lines makes this pattern a little less
strict, but based on the rest of the pattern and the output here, i
personally think this is fine
after carefully verifying this works with the current state of things, i
went ahead and finished the release with this change and it worked just
fine. instead, this PR proposes a way to fix things going forward
fixes#10404
unfortunately, exactly what `python setup.py clean` did doesn't seem
well documented so i dug into the code with a debugger. executing the
`clean` subcommand is done by [this
code](9cc2f5c05c/setuptools/_distutils/command/clean.py (L54-L77))
where the relevant build variables are set by the `build` subcommand
[here](9cc2f5c05c/setuptools/_distutils/command/build.py (L52))
and
[here](9cc2f5c05c/setuptools/_distutils/command/build.py (L112)).
it turns out us running `python setup.py clean` was already redundant
with `rm -rf build` on the next line
i built two releases, one on the latest commit in this PR and another on
44f1dd677b
before the switch to `python -m build`. a simple diff of the resulting
tarballs and wheels fails, presumably because of metadata differences,
but after untaring or unzipping the files, the contents are identical
for all of our built packages
It's a [drop-in
replacement](https://docs.astral.sh/uv/pip/compatibility/) that speeds
things up. I don't see any reason why not.
`--use-pep517` is [set by default](
https://docs.astral.sh/uv/pip/compatibility/#pep-517-build-isolation),
so we don't need it.
`--disable-pip-version-check` also does nothing on uv.
`uv` [uses
separate](https://docs.astral.sh/uv/pip/compatibility/#build-constraints)
`UV_BUILD_CONSTRAINT` and `UV_CONSTRAINT`. I just added it to both to do
the simplest thing here. We could split them.
We probably don't actually need to pipstrap pip anymore, I could take
that out.
What's happening with `parsedatetime` and `python-digitalocean` is that
they were always secretly wrong. Since `pip` compiles bytecode by
default, it was suppressing the errors. If you add the
`--compile-bytecode` flag to `uv`, it passes, but I don't think we
should do that. You can see the failure happen on main by passing
`--no-compile` to the pip args and running `certbot -r -e oldest`.
Now what I don't understand is that some places seem to say the `'\/'`
error from `parsedatetime` only started in python 3.12, whereas others
see it on earlier python. Perhaps pytest is vendorizing python or
something. Not too worried about that, needed to get updated anyway, and
it's an accurate oldest version based on our oldest OSes.
`python-digitalocean` is techincally newer than debian 11, but we've
made that decision before so it seems fine to me.
Part of https://github.com/certbot/certbot/issues/10403
We were never actually updating the versions in certbot-ci and letstest.
Not that it really matters, but let's do that there as well.
Final part of https://github.com/certbot/certbot/issues/10403
I tested running `tools/snap/generate_dnsplugins_snapcraft.sh
certbot-dns-dnsimple` and it put the correct description in to the
`snapcraft.yaml` file.
This is not necessarily the absolute minimum versions/pins we could use,
but it does get tests working. Fixes
https://github.com/certbot/certbot/issues/10418.
---------
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
this is in response to
https://github.com/certbot/certbot/pull/10399#issuecomment-3166305086
this PR does two things:
1. it clarifies what is meant by "build dependencies" in DESIGN.md
2. fixes our workaround for
https://github.com/python-poetry/poetry/issues/4103 which broke when we
moved most of our code under `src` directories. i kept the previous `rm
-rf ${REPO_ROOT}/*/*.egg-info` line around for `letstest` and to
hopefully add some robustness for us if we ever move our code around
again
Alternative implementation for #7908.
In this PR:
- set up ruff in CI (add to `tox.ini`, mark dep in `certbot/setup.py`)
- add a `ruff.toml` that ignores particularly annoying errors. I think
line length isn't actually necessary to set with this workflow since
we're not checking it but putting it there for future usage.
- either fix or ignore the rest of the errors that come with the default
linting configuration. fixed errors are mostly unused variables. ignored
are usually where we're doing weird import things for a specific reason.
blast from the past! resurrects
https://github.com/certbot/certbot/pull/9803 with all of @bmw's changes.
i figured instead of force-pushing a basically brand new branch and
obliterating the old review, i'd just start from a clean slate
fixes#8272
---------
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Co-authored-by: Brad Warren <bmw@eff.org>
Another token of gratitude for a super useful tool and service.
More about codespell: https://github.com/codespell-project/codespell .
I personally introduced it to dozens if not hundreds of projects already
and so far only positive feedback.
CI workflow has 'permissions' set only to 'read' so also should be safe.
---------
Signed-off-by: Yaroslav O. Halchenko <debian@onerussian.com>
Fixes#10252.
See further discussion here: https://github.com/pypa/pip/issues/11457
We are doing option:
> Alternatively, enable the --use-pep517 pip option, possibly with
--no-build-isolation. The --use-pip517 flag will force pip to use the
modern mechanism for editable installs. --no-build-isolation may be
needed if your project has build-time requirements beyond setuptools and
wheel. By passing this flag, you are responsible for making sure your
environment already has the required dependencies to build your package.
Once the legacy mechanism is removed, --use-pep517 will have no effect
and will essentially be enabled by default in this context.
Major changes made here include:
- Add `--use-pep517` to use the modern mechanism, which will be the only
mechanism in future pip releases
- Change to `/src` layout to appease mypy, and because for editable
installs that really is the normal way these days.
- `cd acme && mkdir src && mv acme src/` etc.
- add `where='src'` argument to `find_packages` and add
`package_dir={'': 'src'},` in `setup.py`s
- update `MANIFEST.in` files with new path locations
- Update our many hardcoded filepaths
- Update `importlib-metadata` requirement to fix
double-plugin-entry-point problem in oldest tests
This is a stopgap measure until we upgrade to the newer (but
backwards-incompatible) versions of cloudflare's python library (see
#9938)
---------
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
fixes#10105
this PR updates our minimally required cryptography and pyopenssl
versions as well as updating our policy for choosing minimum dependency
versions generally
before this PR, we were trying to keep compatibility with packages
available in EPEL 9 using the python 3 version available in RHEL 9.
after the discussion in #10105 we decided not to do this anymore
because:
* EPEL 9 may not want to update to certbot 3.0+ anyway because of our
backwards incompatible changes from certbot 2.x
* RHEL 9 appstream repos now contain newer versions of many of our
dependencies for newer versions of python
* alternate installation methods for RHEL 9 based users including our
snaps and pip are available
on a call we then discussed what distro repositories we should track
instead of EPEL 9. our docs previously said Debian sid/unstable, but we
felt this as unnecessary because Debian sid can and does change very
quickly. if we wanted a new dependency there, Debian could probably
accommodate it
we also considered RHEL 10 + EPEL 10, however, these repos are not even
stable yet and certbot and many of its dependencies are not yet packaged
there at all
for these reasons, plus many of the reasons we decided to upgrade past
EPEL 9 with the default python 3 version there, we decided that at least
for now, we will remove any linux distro considerations when choosing
minimal dependency versions of certbot
as i wrote in the contributing docs, we may choose to reconsider this
plan if there are requests for us to do so, but based on the information
above, we are not sure this will ever even happen and removing this
constraint significantly simplifies development of certbot
this hopefully at least helps the problem hit at
https://github.com/certbot/certbot/pull/10126#discussion_r1909714276
i took this approach because in my experience, linux specific shell
commands have crept into our scripts repeatedly over the years so i
think just having macOS devs use the linux versions is much more
reliable. it's what i've personally been doing for years now
when reviewing https://github.com/certbot/certbot/pull/10126 and running
`tools/pinning/oldest/repin.sh` using a freshly created dev environment,
i was repeatedly given the message
> The "poetry.dev-dependencies" section is deprecated and will be
removed in a future version. Use "poetry.group.dev.dependencies"
instead.
i believe this section was generated automatically by poetry's tooling
when it created the initial boilerplate file for us, but we don't use
it, so i just deleted the section which makes the warnings disappear
this PR hopefully improves two things that i hit while working on #10035
1) i found that repinning our dependencies took ~6 minutes!
digging into it a bit, the biggest culprit i found was the inclusion of
`--no-cache-dir` here which seemed to cause poetry to redownload the
same packages over and over in a single run. this comes from
https://github.com/certbot/certbot/pull/9453 which fixed a problem i
(but not alex) was having with a major performance penalty. i removed
the flag here and instead included instructions on clearing poetry's
caches in case anyone ever hits this in the future. with this change,
the script now takes about 40 seconds on my laptop
2) every run of this script ended with the output:
```
Warning: poetry-plugin-export will not be installed by default in a
future version of Poetry.
In order to avoid a breaking change and make your automation
forward-compatible, please install poetry-plugin-export.
explicitly. See https://python-poetry.org/docs/plugins/#using-plugins
for details on how to install a plugin.
To disable this warning run 'poetry config warnings.export false'.
```
setting `POETRY_WARNINGS_EXPORT=false` fixes this which i believe is
safe to do because of
2c8609464c/certbot/setup.py (L53-L56)
i hit this when working on https://github.com/certbot/certbot/pull/10076
where i found that updating all our dependencies no longer worked
because of new deprecations in pyopenssl. this pr fixes that
There are a quite a lot of imports that are unused.
F541 is Unnecessary f-interpolation without placeholders
E711 is incorrect use of == for boolean and None comparisons
## Pull Request Checklist
- [x] The Certbot team has recently expressed interest in reviewing a PR
for this. If not, this PR may be closed due our limited resources and
need to prioritize how we spend them.
- [ ] If the change being made is to a [distributed
component](https://certbot.eff.org/docs/contributing.html#code-components-and-layout),
edit the `main` section of `certbot/CHANGELOG.md` to include a
description of the change being made.
- [ ] Add or update any documentation as needed to support the changes
in this PR.
- [x] Include your name in `AUTHORS.md` if you like.
---------
Co-authored-by: Mads Jensen <atombrella@users.noreply.github.com>
