It looks like we're currently documenting functions that are marked private (prefixed with an underscore) such as https://certbot.eff.org/docs/api/certbot.crypto_util.html#certbot.crypto_util._load_cert_or_req. I do not think we should do this because the functionality is private, should not be used, and including it in our docs just adds visual noise.
This PR stops us from documenting private code and fixes up `tools/sphinx-quickstart.sh` so we don't document it in future modules.
* Do not document private code.
* Don't document private members in the future.
Part of #7550
This PR makes appropriate corrections to run pylint on Python 3.
Why not keeping the dependencies unchanged and just run pylint on Python 3?
Because the old version of pylint breaks horribly on Python 3 because of unsupported version of astroid.
Why updating pylint + astroid to the latest version ?
Because this version only fixes some internal errors occuring during the lint of Certbot code, and is also ready to run gracefully on Python 3.8.
Why upgrading mypy ?
Because the old version does not support the new version of astroid required to run pylint correctly.
Why not upgrading mypy to its latest version ?
Because this latest version includes a new typshed version, that adds a lot of new type definitions, and brings dozens of new errors on the Certbot codebase. I would like to fix that in a future PR.
That said so, the work has been to find the correct set of new dependency versions, then configure pylint for sane configuration errors in our situation, disable irrelevant lintings errors, then fixing (or ignoring for good reason) the remaining mypy errors.
I also made PyLint and MyPy checks run correctly on Windows.
* Start configuration
* Reconfigure travis
* Suspend a check specific to python 3. Start fixing code.
* Repair call_args
* Fix return + elif lints
* Reconfigure development to run mainly on python3
* Remove incompatible Python 3.4 jobs
* Suspend pylint in some assertions
* Remove pylint in dev
* Take first mypy that supports typed-ast>=1.4.0 to limit the migration path
* Various return + else lint errors
* Find a set of deps that is working with current mypy version
* Update local oldest requirements
* Remove all current pylint errors
* Rebuild letsencrypt-auto
* Update mypy to fix pylint with new astroid version, and fix mypy issues
* Explain type: ignore
* Reconfigure tox, fix none path
* Simplify pinning
* Remove useless directive
* Remove debugging code
* Remove continue
* Update requirements
* Disable unsubscriptable-object check
* Disable one check, enabling two more
* Plug certbot dev version for oldest requirements
* Remove useless disable directives
* Remove useless no-member disable
* Remove no-else-* checks. Use elif in symetric branches.
* Add back assertion
* Add new line
* Remove unused pylint disable
* Remove other pylint disable
* Move acme tests to tests/ directory outside of acme module
* Fix call to messages_test in client_test
* Move test_util.py and testdata/ into tests/
* Update manifest to package tests
* Exclude pycache and .py[cod]
Summary of changes in this PR:
- Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below).
- Make repo root README symlink to `certbot` README.
- Pull tests outside of the distributed module.
- Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery.
- Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments.
- Replace mentions of `.` with `certbot` when referring to packages to install, usually editably.
- Clean up some unused code around executing tests in a different directory.
- Create public shim around main and make that the entry point.
New directory structure summary:
```
repo root ("certbot", probably, but for clarity all files I mention are relative to here)
├── certbot
│ ├── setup.py
│ ├── certbot
│ │ ├── __init__.py
│ │ ├── achallenges.py
│ │ ├── _internal
│ │ │ ├── __init__.py
│ │ │ ├── account.py
│ │ │ ├── ...
│ │ ├── ...
│ ├── tests
│ │ ├── account_test.py
│ │ ├── display
│ │ │ ├── __init__.py
│ │ │ ├── ...
│ │ ├── ... # note no __init__.py at this level
│ ├── ...
├── acme
│ ├── ...
├── certbot-apache
│ ├── ...
├── ...
```
* refactor certbot/ and certbot/tests/ to use the same structure as the other packages
* git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g"
* git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g"
* git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g"
* Remove replacement of certbot into . in install_and_test.py
* copy license back out to main folder
* remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/
* Move README back into main folder, and make the version inside certbot/ a symlink
* symlink certbot READMEs the other way around
* move testdata into the public api certbot zone
* update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests
* certbot version has been bumped down a directory level
* make certbot tests directory not a package and import sibling as module
* Remove unused script cruft
* change . to certbot in test_sdists
* remove outdated comment referencing a command that doesn't work
* Install instructions should reference an existing file
* update file paths in Dockerfile
* some package named in tox.ini were manually specified, change those to certbot
* new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code
* remove other instance of pyargs
* fix up some references in _release.sh by searching for ' . ' and manual check
* another stray . in tox.ini
* fix paths in tools/_release.sh
* Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper
* Create public shim around main and make that the entry point
* without pyargs, tests cannot be run from an empty directory
* Remove cruft for running certbot directly from main
* Have main shim take real arg
* add docs/api file for main, and fix up main comment
* Update certbot/docs/install.rst
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Fix comments in readthedocs requirements files to refer to current package
* Update .[docs] reference in contributing.rst
* Move plugins tests to certbot tests directory
* add certbot tests to MANIFEST.in so packagers can run python setup.py test
* move examples directory inside certbot/
* Move CHANGELOG into certbot, and create a top-level symlink
* Remove unused sys and logging from main shim
* nginx http01 test no longer relies on certbot plugins common test
* acme: re-populate uri in deactivate_authorization
* Use fresh authorizations in dry runs
--dry-run now deactivates 'valid' authorizations if it encounters them
when creating a new order.
Resolves#5116.
* remove unused code
* typo in local-oldest-requirements
* better error handling
* certbot-ci: AUTHREUSE to 100 + unskip dry-run test
* improve test coverage for error cases
* restore newline to local-oldest-requirements.txt
I tried to finish up #7214 by removing the code in acme but we can't really do that until #7478 is resolved which we cannot do until we release 0.40.0.
Since we have to wait, this PR adds deprecation warnings for code that uses the TLS-SNI-01 code or was only used by the long deprecated TLS-SNI-01 code.
I'd like this PR to land before our next release.
* Deprecate more code related to TLS-SNI-01.
* Assert about warning message.
Fixes#7368.
When updating the changelog, I replaced the line about running tests on Python 3.8 because I personally think that support for Python 3.8 is the most relevant information for our users/packagers about our changes in this area.
* List support for Python 3.8.
* Update changelog.
Resolves#4945. First PR in order to address #5116.
* acme: Implement authz deactivation
Resolves#4945
* update AUTHORS and CHANGELOG
* typos in mypy annotations
* formatting: missing newline
* improve test_deactivate_authorization
* improve deactivate_authorization
* test: s/STATUS_INVALID/STATUS_DEACTIVATED/
* simplify dict to keyword argument
* acme: add UpdateAuthorization
* acme: use UpdateAuthorization in deactivate_authz
and add mypy annotation
This allows deactivate_authorization to succeed for both ACME v1
and v2 servers.
Fixes#6974.
This PR removes the fallback that consists in retrying to send the keyAuthorization field during a challenge request in case of malformed request.
* Remove keyAuthorization fallback dump in challenges response
* Correct import
* Add changelog entry
This PR is a part of the tls-sni-01 removal plan described in #6849.
As `acme` is a library, we need to put some efforts to make a decent deprecation path before totally removing tls-sni in it. While initialization of `acme.challenges.TLSSNI01` was already creating deprecation warning, not all cases were covered.
For instance, and innocent call like this ...
```python
if not isinstance(challenge, acme.challenges.TLSSNI01):
print('I am not using this TLS-SNI deprecated stuff, what could possibly go wrong?')
```
... would break if we suddenly remove all objects related to this challenge.
So, I use the _Deprecator Warning Machine, Let's Pacify this Technical Debt_ (Guido ®), to make `acme.challenges` and `acme.standalone` patch themselves, and display a deprecation warning on stderr for any access to the tls-sni challenge objects.
No dev should be able to avoid the deprecation warning. I set the deprecation warning in the idea to remove the code on `0.34.0`, but the exact deprecation window is open to discussion of course.
* Modules challenges and standalone patch themselves to generated deprecation warning when tls-sni related objects are accessed.
* Correct unit tests
* Correct lint
* Update challenges_test.py
* Correct lint
* Fix an error during tests
* Update coverage
* Use multiprocessing for coverage
* Add coverage
* Update test_util.py
* Factor the logic about global deprecation warning when accessing TLS-SNI-01 attributes
* Fix coverage
* Add comment for cryptography example.
* Use warnings.
* Add a changelog
* Fix deprecation during tests
* Reload
* Update acme/acme/__init__.py
Co-Authored-By: adferrand <adferrand@users.noreply.github.com>
* Update CHANGELOG.md
* Pick a random free port.
