This PR is the second part of #6497 to ease the integration, following the new plan propose by @bmw here: #6497 (comment)
This PR creates the module certbot.compat.os, that delegates everything to os, and that will be the safeguard against problematic methods of the standard module. On top of that, a quality check wrapper is called in the lint tox environment. This wrapper calls pylint and ensures that standard os module is no used directly in the certbot codebase.
Finally local oldest requirements are updated to ensure that tests will take the new logic when running.
* Add executable permissions
* Add the delegate certbot.compat.os module, add check coding style to enforce usage of certbot.compat.os instead of standard os
* Load certbot.compat.os instead of os
* Move existing compat test
* Update local oldest requirements
* Import sys
* Update account_test.py
* Update os.py
* Update os.py
* Update local oldest requirements
* Implement the new linter_plugin
* Fix local oldest for nginx
* Remove check coding style
* Update linter_plugin.py
* Add several comments
* Update the setup.py
* Add documentation
* Update acme dependencies
* Update certbot/compat/os.py
* Update docs/contributing.rst
* Update linter_plugin.py
* Handle os.path. Simplify checker.
* Add a comment to a reference implementation
* Update changelog
* Fix module registering
* Update docs/contributing.rst
* Update config and changelog
This PR is a part of the tls-sni-01 removal plan described in #6849.
This PR removes --tls-sni-01-port, --tls-sni-01-address and tls-sni-01/tls-sni options from --preferred-challenges. They are replace by deprecation warning, indicating that these options will be removed soon.
This deprecation, instead of complete removal, is done to avoid certbot instances to hard fail if some automated scripts still use these flags for some users.
Once this PR lands, we can remove completely theses flags in one or two release.
* Remove tls-sni related flags in cli. Add a deprecation warning instead.
* Adapt tests to cli and renewal towards tls-sni flags deprecation
* Add https_port option. Make tls_sni_01_port show a deprecation warning, but silently modify https_port if set
* Migrate last items
* Fix lint
* Update certbot/cli.py
Co-Authored-By: adferrand <adferrand@users.noreply.github.com>
* Ensure to remove all occurences of tls-sni-01
* Remove unused parameter
* Revert modifications on cli-help.txt
* Use logger.warning instead of sys.stderr
* Update the logger warning message
* Remove standalone_supported_challenges option.
* Fix order of preferred-challenges
* Remove supported_challenges property
* Fix some tests
* Fix lint
* Fix tests
* Add a changelog
* Clean code, fix test
* Update CI
* Reload
* No hard date for tls-sni removal
* Remove useless cast to list
* Update certbot/tests/renewal_test.py
Co-Authored-By: adferrand <adferrand@users.noreply.github.com>
* Add entry to the changelog
* Add entry to the changelog
* Remove tls-sni from nginx config
* Add a dedicated configuration to define what is the HTTPS port for this certbot instance.
* Correct some tests
* Reestablish default vhost creation
* Clean tls references for nginx integration tests
* Associate https_port only to tests and nginx
This PR fixes certbot-nginx and relevant tests to make them succeed on Windows.
Next step will be to enable integration tests through certbot-ci in a future PR.
* Fix tests and incompabilities in certbot-nginx for Windows
* Fix lint, fix oldest local dependencies
* flip challenge preference in Nginx
* Fix Nginx tests
* Flip challenge preference in Apache
* Flip challenge preference in standalone
* update changelog
* continue to run with tls-sni in integration tests for coverage
* feat(nginx): add and test new parsing abstractions
* chore(nginx parser): fix mypy and address small comments
* chore(nginx parser): clean up by removing context object
* fix integration test and lint
Stop caching the results of ipv6_info in http01.py. A call to choose_vhosts might change the ipv6 results of later calls. Add tests for this and default_listen_addresses more broadly.
Previously, Nginx did not allow `${` to start a variable name. Now it's allowed to. This means we'll be more permissible than Nginx when people are on older versions of Nginx, but it's unlikely anyone was relying on this to fail in the first place, so that's probably ok.
See https://github.com/certbot/website/pull/348#issuecomment-399257703.
```
$ certbot --help all | grep -C 3 nginx-server-root
nginx:
Nginx Web Server plugin - Alpha
--nginx-server-root NGINX_SERVER_ROOT
Nginx server root directory. (default: /etc/nginx)
--nginx-ctl NGINX_CTL
Path to the 'nginx' binary, used for 'configtest' and
```
```
$ CERTBOT_DOCS=1 certbot --help all | grep -C 3 nginx-server-root
nginx:
Nginx Web Server plugin - Alpha
--nginx-server-root NGINX_SERVER_ROOT
Nginx server root directory. (default: /etc/nginx or
/usr/local/etc/nginx)
--nginx-ctl NGINX_CTL
```
* Show both possible Nginx default server root values in docs
* add test
* check that exactly one server root is in the default
* use default magic
* automatically select among default vhosts if we have a port preference
* ports should be strings in the nginx plugin
* clarify port vs preferred_port behavior by adding allow_port_mismatch flag
* update all instances of default_vhosts to all_default_vhosts
* require port
* port should never be None in _get_default_vhost
* Remove unsupported pylint disable options
* star-args removed in Pylint 1.4.3
* abstract-class-little-used removed in Pylint 1.4.3
* Fixes new lint errors
* Copy dummy-variable-rgx expression to new ignored-argument-names expression to ignore unused funtion arguments
* Notable changes
* Refactor to satisfy Pylint no-else-return warning
* Fix Pylint inconsistent-return-statements warning
* Refactor to satisfy consider-iterating-dictionary
* Remove methods with only super call to satisfy useless-super-delegation
* Refactor too-many-nested-statements where possible
* Suppress type checked errors where member is dynamically added (notably derived from josepy.JSONObjectWithFields)
* Remove None default of func parameter for ExitHandler and ErrorHandler
Resolves#5973
* Adds type checking for certbot-nginx
* First pass at type annotation in certbot-nginx
* Ensure linting is disabled for timing imports
* Makes container types specific per PR comments
* Removes unnecessary lint option
* rename delete_default to remove_singleton_listen_params
* update docstring
* add documentation to obj.py
* add test for remove duplicate ipv6only
* Remove ipv6only=on from duplicated vhosts
* add test to make sure ipv6only=on is not erroneously removed
* test default detection
* Allow 'default' along with 'default_server' in Nginx
* Test that default gets written out as default_server in canonical string
* remove superfulous parens
* Refactor _add_directive to separate functions
* UnspacedList isn't idempotent
* refactor parser in add_server_directives and update_or_add_server_directives
* update parser tests
* remove replace=False and add to update_or_add for replace=True in configurator
* remove replace=False and add to update_or_add for replace=True in http01
* update documentation
* feat(nginx plugin): add HSTS enhancement
* chore(nginx): factor out block-splitting code from redirect & hsts enhancements!
* chore(nginx): merge fixes
* address comments
* fix linter: remove a space
* fix(config): remove SSL directives in HTTP block after block split, and remove_directive removes 'Managed by certbot' comment
* chore(nginx-hsts): Move added SSL directives to a constant on Configurator class
* fix(nginx-hsts): rebase on wildcard cert changes
* support wildcards for deploy_cert
* support wildcards for enhance
* redirect enhance and some tests
* update tests
* add display_ops and display_repr
* update display_ops_test and errors found
* say server block
* match redirects properly
* functional code
* start adding tests and lint errors
* add configurator tests
* lint
* change message to be generic to installation and enhancement
* remove _wildcard_domain
* take selecting vhosts out of loop
* remove extra newline
* filter wildcard vhosts by port
* lint
* don't filter by domain
* [^.]+
* lint
* make vhost hashable
* one more tuple
* wrap redirect in if host matches
* return 404 if we've created a new block
* change domain matching to exact match
* insert new redirect directive at the top
* add a redirect block to the top if it doesn't already exist, even if there's an existing redirect
* fix obj tests
* remove active parameter
* update tests
* add back spaces
* move imports
* remove unused code
* get http01 challenge working
* support multiple challenge types in configurator.py
* update existing nginx tests
* lint
* refactor NginxHttp01 and NginxTlsSni01 to both now inherit from NginxChallengePerformer
* remove TODO
* challenges_test tests with both tlssni01 and http01
* Make challenges.py more abstract to make lint happier
* add pylint disables to the tests to make pylint happier about the inheritance and abstraction situation
* no need to cover raise NotImplementedError() lines
* python3 compatibility
* test that http01 perform is called
* only remove ssl from addresses during http01
* Initialize addrs_to_add
* Change Nginx http01 to modify server block so the site doesn't stop serving while getting a cert
* pass existing unit tests
* rename sni --> http01 in unit tests
* lint
* fix configurator test
* select an http block instead of https
* properly test for port number
* use domains that have matching addresses
* remove debugger
* remove access_log and error_log cruft that wasn't being executed
* continue to return None from choose_redirect_vhost when create_if_no_match is False
* add nginx integration test
* create_new_vhost_from_default --> duplicate_vhost
* add source_path property
* set source path for duplicated vhost
* change around logic of where making ssl happens
* don't add listen 80 to newly created ssl block
* cache vhosts list
* remove source path
* add redirect block if we created a new server block
* Remove listen directives when making server block ssl
* Reset vhost cache on parser load
* flip connected pointer direction for finding newly made server block to match previous redirect search constraints
* also test for new redirect block styles
* fix contains_list and test redirect blocks
* update lint, parser, and obj tests
* reset new vhost (fixing previous bug) and move removing default from addrs under if statement
* reuse and update newly created ssl server block when appropriate, and update unit tests
* append newly created server blocks to file instead of inserting directly after, so we don't have to update other vhosts' paths
* add coverage for NO_IF_REDIRECT_COMMENT_BLOCK
* add coverage for parser load calls
* replace some double quotes with single quotes
* replace backslash continuations with parentheses
* update docstrings
* switch to only creating a new block on redirect enhancement, including removing the get_vhosts cache
* update configurator tests
* update obj test
* switch delete_default default for duplicate_vhost
The reason for this PR is many bug fixes in the nginx plugin for changes we haven't released yet are included in #5220 which may not make our next release. If it doesn't, we will (mostly) revert the nginx plugin back to its previous state to avoid releasing these bugs and will revert this PR after the release.
* Revert "Nginx IPv6 support (#5178)"
This reverts commit 68e37b03c8.
* Revert "Fix bug that stopped nginx from finding new server block for redirect (#5198)"
This reverts commit e2ab940ac0.
* Revert "Nginx creates a vhost block if no matching block is found (#5153)"
This reverts commit 95a7d45856.
* Use pipstrap to install a good version of pip
* Use pytest in cb-auto tests
* Remove nose usage in auto_test.py
* remove nose dev dep
* use pytest in test_tests
* Use pytest in tox
* Update dev dependency pinnings
* remove nose multiprocess lines
* Use pytest for coverage
* Use older py and pytest for old python versions
* Add test for Error.__str__
* pin pytest in oldest test
* Fix tests for DNS-DO plugin on py26
* Work around bug for Python 3.3
* Clarify dockerfile comments
* Nginx IPv6 support
* Test and lint fixes
* IPv6 tests to Nginx plugin
* Make ipv6_info() port aware
* Named tuple values for readability
* Lint fix
* Requested changes
* Allow authentication if there's no appropriate vhost
* Update test
* add flag to suppress raising error if no match is found
* Allow installation if there's no appropriate vhost
* remove traceback
* make new vhost ssl
* Fix existing bugs in nginxparser.py and obj.py
* Switch isinstance(x, str) to isinstance(x, six.string_types) in the Nginx plugin
* remove unused import
* remove unneeded custom copy from Addr
* Add docstring for create_new_vhost_from_default
* add test for create_new_vhost_from_default
* add configurator tests and leave finding the first server block for another PR
* don't assume order from a set
* address multiple default_server problem
* don't add vhosts twice
* update unit tests
* update docstring
* Add logger.info message for using default address in tlssni01 auth
* Rename plugins.common.install_ssl_options_conf to plugins.common.install_version_controlled_file
* Install ssl_dhparams file
* Add installation test
* Add ssl_dhparam option when making a server block ssl
* add install_ssl_dhparams to Installer common plugin class
* Remove redundant code and tests
* update MANIFEST.in
* Change add_server_directives replace=True behavior to attempt to replace, but append on failure to find.
* Remove try/except around add_server_directives
* Revert "Don't save keys/csr on dry run (#4380)"
This reverts commit e034b50363.
* Don't save CSRs and keys during dry run
* Factor out _test_obtain_certificate_common
* Add test_obtain_certificate_dry_run
* Wrap key from make_key in util.Key
* Wrap result from make_csr in util.CSR
* move install_ssl_options_conf functionality to common
* add no cover
* compute current hash instead of saving
* make current hash be computed; switch to list of all canonical hashes
* put message directly into assertion
* don't pass logger
* add docstring
* Add unit tests for certbot.plugins.common.install_ssl_options_conf
Fixes#4559.
* Update options-ssl-nginx.conf in prepare, if it hasn't been modified.
* add previous options-ssl-nginx.conf hashes
* InstallSslOptionsConfTest
* remove .new file and only print warning once
* save digest to /etc/letsencrypt
* add comment reminding devs to update hashes
* add comment and test for sha256sum
* treat hash file as text file because python3
* move constants and rename hidden digest file
* Switch to using include directive for Nginx constants
* remove deprecated comment
* give better error message when attempting to insert an existing directive
* make code more readable
* add docstrings
* allow a duplicated directive if it's identical
* comment out precisely repeated directives
* add comments
