It's a [drop-in
replacement](https://docs.astral.sh/uv/pip/compatibility/) that speeds
things up. I don't see any reason why not.
`--use-pep517` is [set by default](
https://docs.astral.sh/uv/pip/compatibility/#pep-517-build-isolation),
so we don't need it.
`--disable-pip-version-check` also does nothing on uv.
`uv` [uses
separate](https://docs.astral.sh/uv/pip/compatibility/#build-constraints)
`UV_BUILD_CONSTRAINT` and `UV_CONSTRAINT`. I just added it to both to do
the simplest thing here. We could split them.
We probably don't actually need to pipstrap pip anymore, I could take
that out.
What's happening with `parsedatetime` and `python-digitalocean` is that
they were always secretly wrong. Since `pip` compiles bytecode by
default, it was suppressing the errors. If you add the
`--compile-bytecode` flag to `uv`, it passes, but I don't think we
should do that. You can see the failure happen on main by passing
`--no-compile` to the pip args and running `certbot -r -e oldest`.
Now what I don't understand is that some places seem to say the `'\/'`
error from `parsedatetime` only started in python 3.12, whereas others
see it on earlier python. Perhaps pytest is vendorizing python or
something. Not too worried about that, needed to get updated anyway, and
it's an accurate oldest version based on our oldest OSes.
`python-digitalocean` is techincally newer than debian 11, but we've
made that decision before so it seems fine to me.
This uses the new `storage.atomic_rewrite` function to write only the
ARI-related value to the config file, leaving other values the same.
Updates `storage.atomic_rewrite` to handle the case where the
`"renewalparams"` section might be empty (which occurs in the new
unittests), and adds some comments.
Note: `atomic_rewrite` is mocked out as a no-op in
`test_resilient_ari_directory_fetches` and `test_renew_before_expiry`
because those tests have a simplistic config object on their
mock_renewable_cert that doesn't include a filename. If we try to write
the config in those tests we'd get an error (trying to write to `None`).
Since those tests aren't intended to test the "store and obey
Retry-After" behavior, I figured it's reasonable to skip it in the name
of testing; but of course, open to idea about the best way to navigate
this.
Part of #10355
Part of https://github.com/certbot/certbot/issues/10403
We were never actually updating the versions in certbot-ci and letstest.
Not that it really matters, but let's do that there as well.
Final part of https://github.com/certbot/certbot/issues/10403
I tested running `tools/snap/generate_dnsplugins_snapcraft.sh
certbot-dns-dnsimple` and it put the correct description in to the
`snapcraft.yaml` file.
Part of https://github.com/certbot/certbot/issues/10403.
As far as I can tell, "stick it in setup.py" is the official way of
handling complex dependencies. But since the version is static, we have
a little more choice here than we had with `certbot/pyproject.toml`.
We could put the version in the respective `pyproject.toml`s and read it
directly from the toml file with something like
[this](https://stackoverflow.com/a/78082561). Or otherwise load and
parse that file. The benefit of doing it that way is that all
non-certbot versions would be canonically in the `pyproject.toml`, and
also if we wanted we could use that same toml parsing to change the
version at release time instead of `sed`. I actually suspect `acme`,
`certbot-ci`, and `certbot-compatibility-test` will be the only ones
where we can completely delete `setup.py`, as the others all have
lockstep dependencies. (side note - we just never update `certbot-ci`
version. it's still set at `0.32.0.dev0`. there's no way this matters
but just noting.) I chose to do it this way instead because it seems
cleaner since we have to keep `setup.py` around anyway, but I don't have
a strong preference.
Based on what I've read, there's not actually a clean way to grab and
insert the version number within the toml file. This is due to [design
decisions](https://github.com/toml-lang/toml/issues/77) by the toml
authors. The clean `all` extras specification that we used in
`certbot/pyproject.toml` [seems to be an
outlier](https://github.com/pypa/setuptools/discussions/3627#discussioncomment-6476654)
because it's pip handling the self-reference, not toml.
This was causing oldest tests to fail on my mac, which has an open file
limit of 256. Locks were being released at exit, but there were more
than 256 tests being run at once. Holding onto the file descriptor for
temporary files was making us keep the files open.
I also removed unnecessary setUps and tearDowns in subclasses so that
this could be fixed in only one spot.
If you wanted to do any testing locally, I was throwing this in places:
```
import errno, os, resource
open_file_handles = []
for fd in range(resource.getrlimit(resource.RLIMIT_NOFILE)[0]):
try: os.fstat(fd)
except OSError as e:
if e.errno == errno.EBADF: continue
open_file_handles.append(fd)
print(f'location description: {len(open_file_handles)}')
```
This is not necessarily the absolute minimum versions/pins we could use,
but it does get tests working. Fixes
https://github.com/certbot/certbot/issues/10418.
---------
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Alternative to https://github.com/certbot/certbot/pull/10408/ and
https://github.com/certbot/certbot/pull/10415/ that fixes production
code for account meta and puts autouse fixtures in certbot and acme
tests. Overrides all `time.sleep` calls while we're at it.
Fixes the production code where it's simple/clean, and fixes the tests
for HTTPServer-based code because we just don't have that many mac users
using standalone.
The responsibility for atomic updates to a config file was previously
spread across different functions. This moves the atomic update to the
lowest level of the call graph.
Also, factor out the code that creates a ConfigObj based on various
inputs (archive dir, cert locations, and renewal params). This allows
cleanly reusing it across the "update" and "create new" paths.
Now the "create new config" code path doesn't have to do any renaming,
and the "update config" code path can assume the input file exists (and
error if it doesn't).
This will make it easier and cleaner to reuse the config-writing code in
#10377.
Part of #10355
this is the final part of
https://github.com/certbot/certbot/issues/10195. this fixes
https://github.com/certbot/certbot/issues/10195
the changes in the first commit were done automatically with the
command:
```
ruff check --fix --extend-select UP006 --unsafe-fixes
```
the second commit configures ruff to check for this to avoid regressions
thanks for bearing with me thru these somewhat large automatically
generated PRs ohemorange 🙏
this is another part of https://github.com/certbot/certbot/issues/10195
these changes were done automatically with the command:
```
ruff check --fix --extend-select UP006 --unsafe-fixes certbot-apache
```
This PR is modeled on https://github.com/certbot/certbot/pull/10373/,
and is part of https://github.com/certbot/certbot/issues/10183.
relevant requirements:
use_tls13 >= 1.13.0
session_tix_off implemented: nginx >= 1.5.9 and openssl_version >=
1.0.2l
session tix off by default: >= 1.23.2
oldest non-deprecated major distros nginx versions:
debian 11 1.18.0
epel 8 1.23.1
ubuntu 22.04 1.18.0
Therefore, we can stop testing for use tls 13 and session tix off
allowed, with the same caveat as [in this
comment](https://github.com/certbot/certbot/pull/10373#issuecomment-3134101604).
While we could add a new split for configs that don't require session
tickets off to be set explicitly since it's the default, I don't think
it's worth doing now. I added a note in the comments about this.
---------
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
this is another part of https://github.com/certbot/certbot/issues/10195
the first commit was done automatically with the command:
```
ruff check --fix --extend-select UP006 --unsafe-fixes certbot/src/certbot/_internal
```
this was unfortunately insufficient as it left a line in webroot.py with
over 100 characters. i fixed this manually in my second commit
this is another part of https://github.com/certbot/certbot/issues/10195
these changes were done automatically with the commands:
```
ruff check --fix --extend-select UP006 --unsafe-fixes certbot
git checkout certbot/src/certbot/_internal
```
fixing up the files under certbot/src/certbot/_internal will be done in
another PR
This sets up a `pyproject.toml` file for certbot, initially generated
[using](https://hatch.pypa.io/latest/intro/#existing-project) `hatch new
--init` and modifying from there. Since we dynamically require acme of a
matching version, I kept that around in `setup.py` to do the simplest
thing in this PR.
Other possible (future) implementations:
- setuptools has a beta implementation to read from a
`requirements.txt`. we could generate one of those.
- we could just hardcode it and update at release time. I like not
having to keep the version up to date in various places but maybe it's
actually fine
- something something integration with poetry pinning?
I think the syntax for setting version dynamically in `pyproject.toml`
is much nicer than what we do in `setup.py`. It's a little silly to do
it there after we've bothered to calculate it, but I put it there in the
hopes of being able to remove it from `setup.py` someday/somehow.
It would be nice to access the version dynamically set in
`pyproject.toml` in `setup.py`, but I do not think it is likely to be
possible.
Here are some useful links regarding this migrations:
- [How to modernize a `setup.py` based
project?](https://packaging.python.org/en/latest/guides/modernize-setup-py-project/)
- [Writing your
`pyproject.toml`](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/)
- [Configuring setuptools using `pyproject.toml` files](
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html)
- [`pyproject.toml`
specification](https://packaging.python.org/en/latest/specifications/pyproject-toml/)
- [Platform compatibility
tags](https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/)
this is in response to
https://github.com/certbot/certbot/pull/10399#issuecomment-3166305086
this PR does two things:
1. it clarifies what is meant by "build dependencies" in DESIGN.md
2. fixes our workaround for
https://github.com/python-poetry/poetry/issues/4103 which broke when we
moved most of our code under `src` directories. i kept the previous `rm
-rf ${REPO_ROOT}/*/*.egg-info` line around for `letstest` and to
hopefully add some robustness for us if we ever move our code around
again
Alternative implementation for #7908.
In this PR:
- set up ruff in CI (add to `tox.ini`, mark dep in `certbot/setup.py`)
- add a `ruff.toml` that ignores particularly annoying errors. I think
line length isn't actually necessary to set with this workflow since
we're not checking it but putting it there for future usage.
- either fix or ignore the rest of the errors that come with the default
linting configuration. fixed errors are mostly unused variables. ignored
are usually where we're doing weird import things for a specific reason.
Part of #10355.
This allows combining the NamespaceConfig object with a cache of ACME
clients, so we don't have to make the whole large NamespaceConfig object
available all the way down the renewal call stack.
The new AriClientPool is responsible for instantiating ACME clients for
the purpose of ARI fetching. It provides only a simple user agent,
listing the Certbot version. The only CLI flag it observes is
--no-verify-ssl.
i wrote this in response to erica's thread at
https://opensource.eff.org/eff-open-source/pl/xtuemgdti78xfx1hn9jwbrfrjy
this hopefully helps some, but i think our logic here is unfortunately
fairly complicated which is reflected in the code comments. feel free to
suggest alternate wording or even just close this if you think our
comments currently in main are good enough
this fixes https://github.com/certbot/certbot/issues/10176 and fixes
https://github.com/certbot/certbot/issues/10257. it is based on
https://github.com/certbot/certbot/pull/10017 and ohemorange said it's
fine for me to cherry-pick their commit here
this change accomplishes a few things:
* because PYTHONPATH is no longer set to
`"$SNAP/lib/python3.12/site-packages:${PYTHONPATH}"` which evaluates to
`"$SNAP/lib/python3.12/site-packages:"` if PYTHONPATH wasn't previously
set, Certbot no longer searches for Python modules in the current
working directory which was causing #10176. i was able to reproduce this
problem with our currently released snap and verify that this change
fixes that problem
* since we no longer set PYTHONPATH at all, it won't be set in user
hooks which was causing https://github.com/certbot/certbot/issues/10257
* as an added bonus, scripts that start with `#!/usr/bin/env
/snap/certbot/current/bin/python3` as suggested
[here](https://github.com/certbot/certbot/issues/10257#issuecomment-2809421320)
are still able to find and import certbot and its dependencies so those
scripts should continue to work. i verified this is the case with manual
testing
finally, i created two news fragments here based on the text at
https://towncrier.readthedocs.io/en/stable/tutorial.html#creating-news-fragments
which says
> You can associate multiple issue numbers with a news fragment by
giving them the same contents.
when run on this PR `towncrier --draft` outputs:
```
Loading template...
Finding news fragments...
Rendering news fragments...
Draft only -- nothing has been written.
What is seen below is what would be written.
## 4.2.0.dev0 - 2025-07-31
### Changed
- Catches and ignores errors during the directory fetch for ARI checking so
that these errors do not hinder the actual certificate issuance.
([#10342](https://github.com/certbot/certbot/issues/10342))
- Removed the dependency on `pytz`.
([#10350](https://github.com/certbot/certbot/issues/10350))
### Fixed
- The Certbot snap no longer sets the environment variable PYTHONPATH stopping
it from picking up Python files in the current directory and polluting the
environment for Certbot hooks written in Python.
([#10176](https://github.com/certbot/certbot/issues/10176),
[#10257](https://github.com/certbot/certbot/issues/10257))
- Previously, we claimed to set FAILED_DOMAINS and RENEWED_DOMAINS env
variables for use by post-hooks when certificate renewals fail, but we were
not actually setting them. Now, we are.
([#10259](https://github.com/certbot/certbot/issues/10259))
- Added `--eab-hmac-alg` parameter to support custom HMAC algorithm for
External Account Binding.
([#10281](https://github.com/certbot/certbot/issues/10281))
- Certbot now always uses the server value from the renewal configuration file
for ARI checks instead of the server value from the current invocation of
Certbot. This helps prevent ARI requests from going to the wrong server if
the user changes CAs.
([#10339](https://github.com/certbot/certbot/issues/10339))
```
---------
Co-authored-by: Erica Portnoy <erica@eff.org>
blast from the past! resurrects
https://github.com/certbot/certbot/pull/9803 with all of @bmw's changes.
i figured instead of force-pushing a basically brand new branch and
obliterating the old review, i'd just start from a clean slate
fixes#8272
---------
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Co-authored-by: Brad Warren <bmw@eff.org>
fixes https://github.com/certbot/certbot/issues/10336 and fixes
https://github.com/certbot/certbot/issues/10357 using the plan at
https://github.com/certbot/certbot/issues/10336#issuecomment-3109192677
while this PR makes the renewal_time function slightly less nice, i
think us catching and handling the exceptions in certbot makes the most
sense so we can do exactly what we want around terminal and file logging
with this change, a output from a failed `sudo certbot renew` run looks
like
```
$ sudo certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/example.org.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
An error occurred requesting ACME Renewal Information (ARI). If this problem persists and you think it's a bug in Certbot, please open an issue at https://github.com/certbot/certbot/issues/new/choose.
Certificate not yet due for renewal
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
The following certificates are not due for renewal yet:
/etc/letsencrypt/live/example.org/fullchain.pem expires on 2025-10-23 (skipped)
No renewals were attempted.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
```
`sudo certbot renew -q` produces no output and the relevant messages in
the log file look like:
```
2025-07-30 19:51:13,267:WARNING:certbot._internal.renewal:An error occurred requesting ACME Renewal Information (ARI). If this problem persists and you think it's a bug in Certbot, please open an issue at https://github.com/certbot/certbot/issues/new/choose.
2025-07-30 19:51:13,267:DEBUG:certbot._internal.renewal:Error while requesting ARI was:
Traceback (most recent call last):
File "/home/brad/certbot/acme/src/acme/client.py", line 366, in renewal_time
raise ValueError('im some error')
ValueError: im some error
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/brad/certbot/certbot/src/certbot/_internal/renewal.py", line 351, in _ari_renewal_time
return acme.renewal_time(cert_pem)[0]
File "/home/brad/certbot/acme/src/acme/client.py", line 370, in renewal_time
raise errors.ARIError(error_msg, now + default_retry_after) from e
acme.errors.ARIError: ('failed to fetch renewal_info URL https://acme-staging-v02.api.letsencrypt.org/acme/renewal-info/oXQaBm1Qt4YtSizBfrSNiElszRY.LMjTHFS4HPbSRMOzLrA9OZId', datetime.datetime(2025, 7, 31, 1, 51, 13, 267088))
```
something weird happened to the changelog in
https://github.com/certbot/certbot/pull/10319. a 4.2.0 entry was added
below the entry for `5.0.0 - main` despite 4.2.0 not having been
released. since it's sounding like we're expecting our next release to
be 4.2.0 and not 5.0, i merged these two changelog entries into one for
4.2.0
i also modified our setup.py files to use 4.2.0.dev0 instead of
5.0.0.dev0 altho this isn't strictly necessary because our release
script will automatically set all version numbers to whatever version we
give it on the command line before building the release
Part of #10183
> Option 4. Stop updating old files with security improvements. If
people want to be on old software they can but then they're not getting
the nice new things. We can either warn or not warn if we see people
using them, either on certbot install (what, who's installing new
certbot on these machines), new cert, cert renewal, or certbot update.
The second two would require code changes, I'm pretty sure. I don't
think we should warn too often because that's how we get people to
silence all output. This is a little weird because we don't usually keep
around deprecated things. We could also warn loudly and see if people
complain. Or do some sort of brownout.
This PR warns every time certbot is run. We could make it run less often
(only when a new config file is installed, probably), but that's a more
extensive code change, and honestly I think it's probably fine? But I
can change it.
Fixes https://github.com/certbot/certbot/issues/10342
When doing ARI checks in acme.renewal_time, we catch RequestException
and return a default value. That's so an unavailable ARI server doesn't
cause issues.
Before we get to acme.renewal_time, we have to create an ACME client,
and in the process fetch a directory. We should make the directory fetch
similarly resilient.
---------
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
