* tests: add certbot-dns-rfc2136 integration tests
* dont use 'with' form of socket.socket
fixes py2 crash
* address some feedback:
- conftest: make DNS server a global resource
- conftest: add dns_xdist parameter into node config
- conftest: add --dns-server=bind flag
- conftest: if configured, point the ACME server to the DNS server
- dnsserver: make it sort-of compatible with xdist (future-proofing)
- context: parameterize dns-rfc2136 credentials file (future proofing)
- context: reduce dns-rfc2136 propagation time to speed up tests
- tox: add a integration-dns-rfc2136 target
- rfc2136: add a test/zone for subdelegation
- rfc2136: skip tests if no DNS server is configured
* try add integration-dns-rfc2136 to CI
* mock recursive dns via RPZ
* update --dns-server args and tox.ini args
* address more feedback:
- dns_server: rename rfc2136 creds file to .tpl
- dns_server: dont vary dns server port, instead we will vary zone names (#8455)
- dns_server: log error if bind9 fails to stop cleanly
- dns_server: replace assert with raise
- context: remove redundant _worker_id
- context: remove redundant cleanup override
- context: fix seek/flush in credentials context manager
- context: rename skip_if_no_server -> ...bind_server
- context: add newline EOF
* conftest: document _setup_primary_node sideeffects
* ci: rfc2136-integration from standard->nightly
* fix _stop_bind (function was renamed to stop)
* ignore errors from shutil.rmtree during cleanup
* dns_server: check for crash while polling
* remove --dry-run from rfc2136 test
* cli: improve Obtaining/Renewing wording
* dont use logger, and use new phrasing
* .display_util.notify: dont wrap
As this function is supposed to be an analogue for print, we do not want
it to wrap by default.
Fixes#7717
This PR adds a `--dns-server` option to the `run_acme_server` test tool, in order to provide an arbitrary DNS server to Pebble or Boulder for the integration tests.
I also take this occasion to make `run_acme_server` a real CLI tool using argparse, and set the `--server-type` (default `pebble`) option as well.
* Set --dns-server flag in run_acme_server
* Default to pebble
* Add documentation
* Configure also Boulder
Do we have any specific reason to run the standard Linux integration tests on Python 2.7?
If not, we should move to a more recent version of Python. This PR does it for Python 3.8.
* tests: fix leaking patch in eff_test.py
* tests: PrintTest->NotifyTest in .display.util
The function was renamed during #8432. This change renames the test as
well.
* IDisplay.notification: add `decorate` param.
The flag allows the caller to control whether the message will be
printed in a decorated way (wrapped by hlines) or in an undecorated
way (similar to print).
It is set to true by default, to reflect the existing behavior of the
function.
* IDisplay.notification: write message to debug log
In the same vein as IReporter, this ensures that all notifications which
are shown to the user also make an appearance in the debug log, which
will aid in troubleshooting.
* restore accidentally deleted newline in decoration
* add helper function for printing status messages
* register: use notify rather than logger
Undoes the change in #8393 in favor of the new helper
* comment .display and ._internal.log
Describing when it is suitable to use each
* add more comments to log.py
* make IDisplay.notification decorate arg private
* rename notify->print and move to .display.util
* rename .display.print back to .display.notify
because linters complain about print being a redefined builtin
* Add a new, simplified version of pipstrap.
* Use tools/pipstrap.py
* Uncomment code
* Refactor pip_install.py and provide hashes.
* Fix test_sdists.sh.
* Make code work on Python 2.
* Call strip_hashes.py using Python 3.
* Pin the oldest version of httplib2 used in distros
* Strip enum34 dependency.
* Remove pip pinnings from dev_constraints.txt
* Correct pipstrap docstring.
* Don't set working_dir twice.
* Add comments
* Remove python_version from mypy.ini.
* Fix magic_typing
* Ignore msvcrt usage.
* make mypy happier
* clean up changes
* Add type for reporter queue
* More mypy fixes
* Fix pyrfc3339 str.
* Remove unused import.
* Make certbot.util mypy work in both Pythons
* Fix typo
While reviewing https://github.com/certbot/certbot/pull/8404, it occurred to me that we're keeping both the generated files and the script used to generate them in `git`. Keeping both around seems unnecessary and is almost asking for the files to get out of sync at some point in the future. I fixed that by removing the files, adding them to `.gitignore`, and updating `build_remote.py` to generate them as needed.
* Remove generated files.
* Add generated files to gitignore.
* Reuse generate_dnsplugins_all.sh in build_remote
While working on https://github.com/certbot/certbot/issues/8400, I noticed our Fedora AMIs are quite out of date. I considered updating them and what we could do to avoid the AMIs becoming so out-of-date in the future, but I think we don't actually need these tests.
I pulled a new count of Certbot users by OS and we have less than 7,000 Fedora users meaning only ~0.26% of Certbot users run Fedora. (I think Fedora is a popular desktop OS, but not as popular of a server OS which is where Certbot normally runs.)
Also, Certbot is regularly updated on Fedora including Fedora Rawhide or the rolling release version of Fedora which is similar to Debian sid/unstable. Rawhide changes far too frequently for it to make sense for us to run tests there in my opinon, but that also means that many problems such as Certbot's unit tests failing to run because of Fedora changes will be caught there by our Fedora maintainers before we'd even see it. This is how https://github.com/certbot/certbot/issues/7106 became an issue and how I learned [Certbot worked on Python 3.9 before we could run tests on it](https://github.com/certbot/certbot/issues/8134#issuecomment-655106169).
Because of all this, I think we should just simplify things and remove these tests. If a problem arises in the future, we can always add them back.
Fixes#8409.
Change the line in the README to allow `sudo /snap/bin/lxd.migrate -yes` to fail (for example, if there's nothing to migrate), but the whole command to succeed.
I tested this on a clean Focal install and confirmed it works.
Fixes https://github.com/certbot/certbot/issues/8400.
I had to switch the package installed in `apacheconftest` to `libapache2-mod-wsgi-py3` because Ubuntu 20.10 removed the Python 2 version of this module.
I didn't add this AMI to `tests/letstest/auto_targets.yaml` because like Ubuntu 20.04, `certbot-auto` has never worked on the OS.
* Add Ubuntu 20.20 test farm tests
* Try Python 3 WSGI
This PR adds the following documentation improvements to fix https://github.com/certbot/certbot/issues/7958:
- Simplify building external plugins
- Separate out certbot snap instructions from plugin instructions
- Mention that dnsimple is just an example for the plugin instructions
- Mention remote build for other architectures
- Mention snap doc exists elsewhere in developer guide (`contributing.rst`)
* Set up generate_dnsplugins_all.sh for all files and parametrize snapcraft and postrefreshhook files
* Create constraints file in the generate_dnsplugins_all script
* Separate out plugin and certbot snaps and update instructions
* Add remote build instructions
* Add pointers to the README to contributing.rst
Fixes#8355
During the troubleshooting of #8355, I came to the conclusion that using buildkit was creating the problem. Without it all docker images are built correctly. Initially buildkit was enabled to avoid a building problem in Azure Pipeline, but I also found in my recent tests that this problem was not there anymore.
You can find more details about the troubleshooting and reasoning in #8355.
As a consequence, I disable the usage of buildkit in this PR which will solve the issue.
Fixes#8202
This PR adds an Azure Pipeline job to execute certbot plugins --prepare for each Docker image created during the CI on amd64.
* Prepare basic integration tests for certbot dockers
* Add a displayName for the integration tests task
* Add timeout to DNS query function calls
* Modify tests to account for new timeout variable
* Add change to CHANGELOG
* Add `dns.exception.Timeout` to exception handler
* Move changelog to 1.10.0
Fixes https://github.com/certbot/certbot/issues/8171.
See the comment at the top of the script to learn how to set things up and run this. Running the script between releases will have no effect on our snaps and it should fail when creating the GitHub release. The latter is described at https://github.com/certbot/certbot/pull/8189#discussion_r466707114.
* Rename create_github_release to finish_release
* Add initial version of snap release automation.
* Handle snapcraft login.
* Catch OSError raised when snapcraft doesn't exist.
* Update documentation.
* Only publish the Certbot snap for now.
* Fix typo.
* Document other exceptions.
* Document assertion
* Add status message before getting revisions.
* Publish all snaps.
