In CentOS 6 default httpd configuration, the `LoadModule ssl_module ...` is handled in `conf.d/ssl.conf`. As the `VirtualHost` configuration files in `conf.d/` are loaded in alphabetical order, this means that all files that have `<IfModule mod_ssl.c>` and are loaded before `ssl.conf` are effectively ignored. This PR moves the `LoadModule ssl_module` to the main `httpd.conf` while leaving a conditional `LoadModule` directive in `ssl.conf`.
Features
- Reads the module configuration from `ssl.conf` in case some modifications to paths have been made by the user.
- Falls back to default paths if the directive doesn't exist.
- Moves the `LoadModule` directive in `ssl.conf` inside `<IfModule !mod_ssl.c>` to avoid printing warning messages of duplicate module loads.
- Adds `LoadModule ssl_module` inside of `<IfModule !mod_ssl.c>` to the top of the main `httpd.conf`.
- Ensures that these modifications are not made multiple times.
Fixes: #6606
* Fix CentOS6 installer issue
* Changelog entry
* Address review comments
* Do not enable mod_ssl if multiple different values were found
* Add test comment
* Address rest of the review comments
* Address review comments
* Better ifmodule argument checking
* Test fixes
* Make linter happy
* Raise an exception when differing LoadModule ssl_module statements are found
* If IfModule !mod_ssl.c with LoadModule ssl_module already exists in Augeas path, do not create new LoadModule directive
* Do not use deprecated assertion functions
* Address review comments
* Kick tests
* Revert "Kick tests"
This reverts commit 967bb574c2.
* Address review comments
* Add pydoc return value to create_ifmod
Add a new test to make sure that we are covering all the branches of get_virtual_hosts() regardless of the order that Augeas returns the found VirtualHost paths.
Fixes: #6813
* Add a test to ensure test coverage regardless of the order of returned vhosts
* Use deepcopy instead, and increase coverage requirement back to 100%
To fix one of the two uncovered lines in certbot-apache, given in #6880. Instead of adding a test to just increase the coverage, this fixes the uncovered line by removing the unused code.
* Add a dedicated configuration to define what is the HTTPS port for this certbot instance.
* Remove tls-sni in apache plugin
* Update constants.py
* Update interfaces.py
* Remove option
* Simplify a test
So merging the study from @bmw and me, here is what happened.
Each invocation of `certbot.logger.post_arg_parse_setup` create a file handler on `letsencrypt.log`. This function also set an atexit handler invoking `logger.shutdown()`, that have the effect to close all logger file handler not already closed at this point. This method is supposed to be called when a python process is close to exit, because it makes all logger unable to write new logs on any handler.
Before #6667 and this PR, for tests, the atexit handle would be triggered only at the end of the pytest process. It means that each test that launches `certbot.logger.post_arg_parse_setup` add a new file handler. These tests were typically connecting the file handler on a `letsencrypt.log` located in a temporary directory, and this directory and content was wipped out at each test tearDown. As a consequence, the file handles, not cleared from the logger, were accumulating in the logger, with all of them connected to a deleted file log, except the last one that was just created by the current test. Considering the number of tests concerned, there were ~300 file handler at the end of pytest execution.
One can see that, on prior #6667, by calling `print(logger.getLogger().handlers` on the `tearDown` of these tests, and see the array growing at each test execution.
Even if this represent a memory leak, this situation was not really a problem on Linux: because a file can be deleted before it is closed, it was only meaning that a given invocation of `logger.debug` for instance, during the tests, was written in 300 log files. The overhead is negligeable. On Windows however, the file handlers were failing because you cannot delete a file before it is closed.
It was one of the reason for #6667, that added a call to `logging.shutdown()` at each test tearDown, with the consequence to close all file handlers. At this point, Linux is not happy anymore. Any call to `logger.warn` will generate an error for each closed file handler. As a file handler is added for each test, the number of errors grows on each test, following an arithmetical suite divergence.
On `test_sdists.py`, that is using the bare setuptools test suite without output capturing, we can see the damages. The total output takes 216000 lines, and 23000 errors are generated. A decent machine can support this load, but a not a small AWS instance, that is crashing during the execution. Even with pytest, the captured output and the memory leak become so large that segfaults are generated.
On the current PR, the problem is solved, by resetting the file handlers array on the logging system on each test tearDown. So each fileHandler is properly closed, and removed from the stack. They do not participate anymore in the logging system, and can be garbage collected. Then we stay on always one file handler opened at any time, and tests can succeed on AWS instances.
For the record, here is all the places where the logging system is called and fail if there is still file handlers closed but not cleaned (extracted from the original huge output before correction):
```
Logged from file account.py, line 116
Logged from file account.py, line 178
Logged from file client.py, line 166
Logged from file client.py, line 295
Logged from file client.py, line 415
Logged from file client.py, line 422
Logged from file client.py, line 480
Logged from file client.py, line 503
Logged from file client.py, line 540
Logged from file client.py, line 601
Logged from file client.py, line 622
Logged from file client.py, line 750
Logged from file cli.py, line 220
Logged from file cli.py, line 226
Logged from file crypto_util.py, line 101
Logged from file crypto_util.py, line 127
Logged from file crypto_util.py, line 147
Logged from file crypto_util.py, line 261
Logged from file crypto_util.py, line 283
Logged from file crypto_util.py, line 307
Logged from file crypto_util.py, line 336
Logged from file disco.py, line 116
Logged from file disco.py, line 124
Logged from file disco.py, line 134
Logged from file disco.py, line 138
Logged from file disco.py, line 141
Logged from file dns_common_lexicon.py, line 45
Logged from file dns_common_lexicon.py, line 61
Logged from file dns_common_lexicon.py, line 67
Logged from file dns_common.py, line 316
Logged from file dns_common.py, line 64
Logged from file eff.py, line 60
Logged from file eff.py, line 73
Logged from file error_handler.py, line 105
Logged from file error_handler.py, line 110
Logged from file error_handler.py, line 87
Logged from file hooks.py, line 248
Logged from file main.py, line 1071
Logged from file main.py, line 1075
Logged from file main.py, line 1189
Logged from file ops.py, line 122
Logged from file ops.py, line 325
Logged from file ops.py, line 338
Logged from file reporter.py, line 55
Logged from file selection.py, line 110
Logged from file selection.py, line 118
Logged from file selection.py, line 123
Logged from file selection.py, line 176
Logged from file selection.py, line 231
Logged from file selection.py, line 310
Logged from file selection.py, line 66
Logged from file standalone.py, line 101
Logged from file standalone.py, line 88
Logged from file standalone.py, line 97
Logged from file standalone.py, line 98
Logged from file storage.py, line 52
Logged from file storage.py, line 59
Logged from file storage.py, line 75
Logged from file util.py, line 56
Logged from file webroot.py, line 165
Logged from file webroot.py, line 186
Logged from file webroot.py, line 187
Logged from file webroot.py, line 204
Logged from file webroot.py, line 223
Logged from file webroot.py, line 234
Logged from file webroot.py, line 235
Logged from file webroot.py, line 237
Logged from file webroot.py, line 91
```
* Reapply #6667
* Make setuptools delegates tests execution to pytest, like in acme module.
* Clean handlers at each tearDown to avoid memory leaks.
* Update changelog
Apache plugin will now use command line default values from `ApacheConfingurator.OS_DEFAULTS` instead of respective distribution override when `CERTBOT_DOCS=1` environment variable is present.
Fixes: #6234
* Apache: respect CERTBOT_DOCS environment variable
* Move the tests to apache plugin
Attempts to configure all of the following VirtualHosts for answering the HTTP challenge:
* VirtualHosts that have the requested domain name in either `ServerName` or `ServerAlias` directive.
* VirtualHosts that have a wildcard name that would match the requested domain name.
This also applies to HTTPS VirtualHosts, making Apache plugin able to handle cases where HTTP redirection takes place in reverse proxy or similar, before reaching the Apache HTTPD.
Even though also HTTPS VirtualHosts are selected, Apache plugin tries to ensure that at least one of the selected VirtualHosts listens to HTTP-01 port (configured with `--http-01-port` CLI option). So in a case where only HTTPS VirtualHosts exist, but user wants to configure those, `--http-01-port` parameter needs to be set for the port configured to the HTTPS VirtualHost(s).
Fixes: #6730
* Select all matching VirtualHosts for HTTP-01 challenges instead of just one
* Finalize PR and add tests
* Changelog entry
This will immediately address the breakage reported in #6682 and tracked at #6685. Pip 19.0.0 and 19.0.1 don't allow commas in filenames, so don't use commas in filenames in certbot-apache test code.
I've confirmed that this fixes the issue on a machine that fails with the version of certbot-auto currently in master: recent version of virtualenv, python 2.7.
Steps to test:
push master to test box
run tools/venv.py
activate venv
pip --version: 19.0.1
pip install ./certbot-apache/: fails
push branch code to test box
confirm pip --version still 19.0.1
pip install ./certbot-apache/: success
* Rename old,default.conf to old-and-default.conf
* Update changelog
* sites-enabled should contain a symlink to sites-available
Fixes#6585.
I wrote up three suggestions for fixing this at https://github.com/certbot/certbot/issues/6585#issuecomment-448054502. I took the middle approach of requiring the user to provide an ACME server to use. I like this better than the other approaches which were:
> Resolve#5938 instead of this issue.
There is value in these tests as is over the compatibility tests in that they don't use Docker and run on different OSes.
> Spin up a local Python server to return the directory object.
Trying to set up a dummy ACME server seemed hacky and error prone.
Other notes about this PR are:
* I put the Pebble setup in `tox.ini` rather than `.travis.yml` as this seems much cleaner and more natural.
* I created a new `tox` environment called `apacheconftest-with-pebble` that reuses the code from `testenv:apacheconftest` so `apacheconftest` can continue to be used with servers other than Pebble like is done in our test farm tests.
* I chose the environment variable `SERVER` for consistency with our integration tests. I chose to not give this environment variable a default but to fail fast when it is not set.
* I ran test farm tests on this PR and they passed.
* flip challenge preference in Nginx
* Fix Nginx tests
* Flip challenge preference in Apache
* Flip challenge preference in standalone
* update changelog
* continue to run with tls-sni in integration tests for coverage
This PR adds two new command line parameters, --apache-ctlpath and --apache-binpath both of which are used to construct commands that we shell out for.
The way that we previously fetched values either from Certbot configuration object or the dictionary of distribution based constants is now also unified, and the active options are parsed in prepare() to make it easier to override needed values for the distributions needing this behavior.
Fixes: #5338
* Added the command line options and parsing
* Refactor existing code
* Distro override updates
* Handle vhost_root from cli
* Fix compatibility tests
* Add comment about changes to command line arguments
* Check None properly
* Made help texts consistent
* Keep the old defaults
* Move to shorter CLI parameter names
* No need for specific bin path, nor apache_cmd anymore
* Make sure that we use user provided vhost-root value
* Fix alt restart commands in overrides
* Fix version_cmd defaults in overrides
* Fix comparison
* Remove cruft, and use configuration object for parser parameter
If user provides a custom --apache-vhost-root path that's not parsed by Apache per default, Certbot fails the challenge validation. While the VirtualHost on custom path is correctly found, and edited, it's still not seen by Apache. This PR adds a temporary Include directive to the root Apache configuration when writing the challenge tokens to the VirtualHost.
interfaces.GenericUpdater and new enhancement interface updater functions get run on every invocation of Certbot with "renew" verb for every lineage. This causes performance problems for users with large configurations, because of plugin plumbing and preparsing happening in prepare() method of installer plugins. This PR moves the responsibility to call prepare() to the plugin (possibly) implementing a new style enhancement interface.
Fixes: #6153
* Do not call IPlugin.prepare() for updaters when running renew
* Check prepare called in tests
* Refine pydoc and make the function name more informative
* Verify the plugin type
* Remove apacheconftest packages.
The apacheconftests handle installing Apache dependencies, so let's remove it from the general case.
* We don't need to run dpkg -s in before_install.
* Remove augeas sources.
We only needed it for Ubuntu Precise which is dead and it doesn't work in Ubuntu Xenial.
* Upgrade Python 3.6 tests to 3.7.
Let's continue the approach of testing on the oldest and newest versions of Python 3. We will continue testing on Python 3.6 in the nightly tests.
* Revert "We don't need to run dpkg -s in before_install."
This reverts commit e5d35099a7.
* let apacheconftest handle deps
* Use greater than or equal to in requirements.
This changes the existing requirements using strictly greater than to greater
than or equal to so that they're more conventional.
* Use >= for certbot-postfix.
Despite it previously saying 'certbot>0.23.0', certbot-postfix/local-oldest-requirements.txt was pinned to 0.23.0 so let's just use certbot>=0.23.0.
This PR adds the functionality to enhance Apache configuration to include HTTP Strict Transport Security header with a low initial max-age value.
The max-age value will get increased on every (scheduled) run of certbot renew regardless of the certificate actually getting renewed, if the last increase took place longer than ten hours ago. The increase steps are visible in constants.AUTOHSTS_STEPS.
Upon the first actual renewal after reaching the maximum increase step, the max-age value will be made "permanent" and will get value of one year.
To achieve accurate VirtualHost discovery on subsequent runs, a comment with unique id string will be added to each enhanced VirtualHost.
* AutoHSTS code rebased on master
* Fixes to match the changes in master
* Make linter happy with metaclass registration
* Address small review comments
* Use new enhancement interfaces
* New style enhancement changes
* Do not allow --hsts and --auto-hsts simultaneuously
* MyPy annotation fixes and added test
* Change oldest requrements to point to local certbot core version
* Enable new style enhancements for run and install verbs
* Test refactor
* New test class for main.install tests
* Move a test to a correct test class
* Remove unneeded sys import.
Once upon a time we needed this in some of these setup.py files because we were
using sys in the file, but we aren't anymore so let's remove the import.
* use setuptools instead of distutils
* Remove unsupported pylint disable options
* star-args removed in Pylint 1.4.3
* abstract-class-little-used removed in Pylint 1.4.3
* Fixes new lint errors
* Copy dummy-variable-rgx expression to new ignored-argument-names expression to ignore unused funtion arguments
* Notable changes
* Refactor to satisfy Pylint no-else-return warning
* Fix Pylint inconsistent-return-statements warning
* Refactor to satisfy consider-iterating-dictionary
* Remove methods with only super call to satisfy useless-super-delegation
* Refactor too-many-nested-statements where possible
* Suppress type checked errors where member is dynamically added (notably derived from josepy.JSONObjectWithFields)
* Remove None default of func parameter for ExitHandler and ErrorHandler
Resolves#5973
* resolved mypy untyped defs in parser.py
* resolved mypy untyped defs in obj.py
* removed unused imports
* resolved mypy untyped defs in http_01.py
* resolved mypy untyped defs in tls_sni_01.py
* resolved mypy untyped defs in configurator.py
* address mypy too-many-arguments error in override_centos.py
* resolved mypy untyped defs in http_01_test.py
* removed unused 'conf' argument that was causing mypy method assignment error
* address mypy error where same variable reassigned to different type
* address pylint and coverage issues
* one character space change for formatting
* fix required acme version for certbot-apache
- Fix code to log separate error messages when either SSLCertificateFile or SSLCertificateKeyFile -
directives are not found.
- Update the section in install.rst where the relevant error is referenced.
- Edit a docstring where 'cert' previously referred to certificate.
- Edit test_deploy_cert_invalid_vhost in the test suite to cover changes.
Fixes#5525.
Fixes#5490.
There's a lot of possibilities discussed in #5490, but I'll try and explain what I actually did here as succinctly as I can. Unfortunately, there's a fair bit to explain. My goal was to break lockstep and give us tests to ensure the minimum specified versions are correct without taking the time now to refactor our whole test setup.
To handle specifying each package's minimum acme/certbot version, I added a requirements file to each package. This won't actually be included in the shipped package (because it's not in the MANIFEST).
After creating these files and modifying tools/pip_install.sh to use them, I created a separate tox env for most packages (I kept the DNS plugins together for convenience). The reason this is necessary is because we currently use a single environment for each plugin, but if we used this approach for these tests we'd hit issues due to different installed plugins requiring different versions of acme/certbot. There's a lot more discussion about this in #5490 if you're interested in this piece. I unfortunately wasted a lot of time trying to remove the boilerplate this approach causes in tox.ini, but to do this I think we need negations described at complex factor conditions which hasn't made it into a tox release yet.
The biggest missing piece here is how to make sure the oldest versions that are currently pinned to master get updated. Currently, they'll stay pinned that way without manual intervention and won't be properly testing the oldest version. I think we should solve this during the larger test/repo refactoring after the release because the tests are using the correct values now and I don't see a simple way around the problem.
Once this lands, I'm planning on updating the test-everything tests to do integration tests with the "oldest" versions here.
* break lockstep between packages
* Use per package requirements files
* add local oldest requirements files
* update tox.ini
* work with dev0 versions
* Install requirements in separate step.
* don't error when we don't have requirements
* install latest packages in editable mode
* Update .travis.yml
* Add reminder comments
* move dev to requirements
* request acme[dev]
* Update pip_install documentation
In `deploy_cert()` and `enhance()`, the user will be presented with a dialog to choose from the VirtualHosts that can be covered by the wildcard domain name. The (multiple) selection result will then be handled in a similar way that we previously handled a single VirtualHost that was returned by the `_find_best_vhost()`.
Additionally the selected VirtualHosts are added to a dictionary that maps selections to a wildcard domain to be reused in the later `enhance()` call and not forcing the user to select the same VirtualHosts again.
* Apache plugin wildcard support
* Present dialog only once per domain, added tests
* Raise exception if no VHosts selected for wildcard domain
* Drop support for EOL Python 2.6
* Use more helpful assertIn/NotIn instead of assertTrue/False
* Drop support for EOL Python 3.3
* Remove redundant Python 3.3 code
* Restore code for RHEL 6 and virtualenv for Py2.7
* Revert pipstrap.py to upstream
* Merge py26_packages and non_py26_packages into all_packages
* Revert changes to *-auto in root
* Update by calling letsencrypt-auto-source/build.py
* Revert permissions for pipstrap.py
Unfortunately, the way that Apache merges the configuration directives is different for mod_rewrite and <Location> / <Directory> directives.
To work around basic auth in VirtualHosts, the challenge override Include had to be split in two. The first part handles overrides for RewriteRule and the other part will handle overrides for <Directory> and <Location> directives.
* Fix docstring quote spacing
* Remove unneeded directives
* Enable mod_rewrite
* Remove ifmod rewrite
* Use stricter rewriterule
* Uncomment tests
* Fix order args
* Remove S which doesn't seem to work across contexts
* Use double backslash to make pylint
* Fix enmod test
* Fix http-01 tests
* Test for rewrite
* check for Include in vhost
* add test_same_vhost
* Don't add includes twice
* Include default vhosts in search
* Respect port in find_best_http_vhost
* Add find_best_http_vhost port test
* Filter by port in http01
* Add a simple version of HTTP01
* remove cert from chall name
* make directory work on 2.2
* cleanup challenges when finished
* import shutil
* fixup perform and cleanup tests
* Add tests for http_01.py
Class inheritance based approach to distro specific overrides.
How it works:
The certbot-apache plugin entrypoint has been changed to entrypoint.ENTRYPOINT which is a variable containing appropriate override class for system, if available.
Override classes register themselves using decorator override.register() which takes a list of distribution fingerprints (ID & LIKE variables in /etc/os-release, or platform.linux_distribution() as a fallback). These end up as keys in dict override.OVERRIDE_CLASSES and values for the keys are references to the class that called the decorator, hence allowing self-registration of override classes when they are imported. The only file importing these override classes is entrypoint.py, so adding new override classes would need only one import in addition to the actual override class file.
Generic changes:
Parser initialization has been moved to separate class method, allowing easy override where needed.
Cleaned up configurator.py a bit, and moved some helper functions to newly created apache_util.py
Split Debian specific code from configurator.py to debian_override.py
Changed define_cmd to apache_cmd because the parameters are for every distribution supporting this behavior, and we're able to use the value to build the additional configuration dump commands.
Moved add_parser_mod() from configurator to parser add_mod()
Added two new configuration dump parsing methods to update_runtime_variables() in parser: update_includes() and update_modules().
Changed init_modules() in parser to accommodate the changes above. (ie. don't throw existing self.modules out).
Moved OS based constants to their respective override classes.
Refactored configurator class discovery in tests to help easier test case creation using distribution based override configurator class.
tests.util.get_apache_configurator() now takes keyword argument os_info which is string of the desired mock OS fingerprint response that's used for picking the right override class.
This PR includes two major generic additions that should vastly improve our parsing accuracy and quality:
Includes are parsed from config dump from httpd binary. This is mandatory for some distributions (Like OpenSUSE) to get visibility over the whole configuration tree because of Include statements passed on in command line, and not via root httpd.conf file.
Modules are parsed from config dump from httpd binary. This lets us jump into correct IfModule directives if for some reason we have missed the module availability (because of one being included on command line or such).
Distribution specific changes
Because of the generic changes, there are two distributions (or distribution families) that do not provide such functionality, so it had to be overridden in their respective override files. These distributions are:
CentOS, because it deliberately limits httpd binary stdout using SELinux as a feature. We are doing opportunistic config dumps here however, in case SELinux enforcing is off.
Gentoo, because it does not provide a way to invoke httpd with command line parsed from its specific configuration file. Gentoo relies heavily on Define statements that are passed over from APACHE2_OPTS variable /etc/conf.d/apache2 file and most of the configuration in root Apache configuration are dependent on these values.
Debian
Moved the Debian specific parts from configurator.py to Debian specific override.
CentOS
Parsing of /etc/sysconfig/httpd file for additional Define statements. This could hold other parameters too, but parsing everything off it would require a full Apache lexer. For CLI parameters, I think Defines are the most common ones. This is done in addition of opportunistic parsing of httpd binary config dump.
Added CentOS default Apache configuration tree for realistic test cases.
Gentoo
Parsing Defines from /etc/conf.d/apache2 variable APACHE2_OPTS, which holds additional Define statements to enable certain functionalities, enabling parts of the configuration in the Apache2 DOM. This is done instead of trying to parse httpd binary configuration dumps.
Added default Apache configuration from Gentoo to testdata, including /etc/conf.d/apache2 file for realistic test cases.
* Distribution specific override functionality based on class inheritance
* Need to patch get_systemd_os_like to as travis has proper os-release
* Added pydoc
* Move parser initialization to a method and fix Python 3 __new__ errors
* Parser changes to parse HTTPD config
* Try to get modules and includes from httpd process for better visibility over the configuration
* Had to disable duplicate-code because of test setup (PyCQA/pylint/issues/214)
* CentOS tests and linter fixes
* Gentoo override, tests and linter fixes
* Mock the process call in all the tests that require it
* Fix CentOS test mock
* Restore reseting modules list functionality for cleanup
* Move OS fingerprinting and constant mocks to parent class
* Fixes requested in review
* New entrypoint structure and started moving OS constants to override classes
* OS constants move continued, test and linter fixes
* Removed dead code
* Apache compatibility test changest to reflect OS constant restructure
* Test fix
* Requested changes
* Moved Debian specific tests to own test file
* Removed decorator based override class registration in favor of entrypoint dict
* Fix for update_includes for some versions of Augeas
* Take fedora fix into account in tests
* Review fixes
* Use pipstrap to install a good version of pip
* Use pytest in cb-auto tests
* Remove nose usage in auto_test.py
* remove nose dev dep
* use pytest in test_tests
* Use pytest in tox
* Update dev dependency pinnings
* remove nose multiprocess lines
* Use pytest for coverage
* Use older py and pytest for old python versions
* Add test for Error.__str__
* pin pytest in oldest test
* Fix tests for DNS-DO plugin on py26
* Work around bug for Python 3.3
* Clarify dockerfile comments
This changes the apache plugin behaviour to only parse enabled configuration files and respecting the --apache-vhost-root CLI parameter for new SSL vhost creation. If --apache-vhost-root isn't defined, or doesn't exist, the SSL vhost will be created to originating non-SSL vhost directory.
This PR also implements actual check for vhost enabled state, and makes sure parser.parse_file() does not discard changes in Augeas DOM, by doing an autosave.
Also handles enabling the new SSL vhost, if it's on a path that's not parsed by Apache.
Fixes: #1328Fixes: #3545Fixes: #3791Fixes: #4523Fixes: #4837Fixes: #4905
* First changes
* Handle rest of the errors
* Test fixes
* Final fixes
* Make parse_files accessible and fix linter problems
* Activate vhost at later time
* Cleanup
* Add a new test case, and fix old
* Enable site later in deploy_cert
* Make apache-conf-test default dummy configuration enabled
* Remove is_sites_available as obsolete
* Cleanup
* Brought back conditional vhost_path parsing
* Parenthesis
* Fix merge leftovers
* Fix to work with the recent changes to new file creation
* Added fix and tests for non-symlink vhost in sites-enabled
* Made vhostroot parameter for ApacheParser optional, and removed extra_path
* Respect vhost-root, and add Include statements to root configuration if needed
* Fixed site enabling order to prevent apache restart error while enabling mod_ssl
* Don't exclude Ubuntu / Debian vhost-root cli argument
* Changed the SSL vhost directory selection priority
* Requested fixes for paths and vhost discovery
* Make sure the Augeas DOM is written to disk before loading new files
* Actual checking for if the file is parsed within existing Apache configuration
* Fix the order of dummy SSL directives addition and enabling modules
* Restructured site_enabled checks
* Enabling vhost correctly for non-debian systems
* Rename plugins.common.install_ssl_options_conf to plugins.common.install_version_controlled_file
* Install ssl_dhparams file
* Add installation test
* Add ssl_dhparam option when making a server block ssl
* add install_ssl_dhparams to Installer common plugin class
* Remove redundant code and tests
* update MANIFEST.in
* Addressing #4071 Wrote an ImmutableReturnMixin to prevent developers overriding return_value in certain Mock objects
* Language
* Loosening the assumption that underlying _mock objects need to be Immutable-like simplifies implementation
* Addressing #4071
* Ensure side_effects and return_values are pushed down to the underlying _mock in FreezableMocks. And IDisplay mocks are no longer frozen in _create_get_utility_mock()
* Edit a handful of tests to not override the mock_get_utility return_value
* Brief explainer of FreezableMock.__setattr__
* Incorporating PR feedback and some compatibility
* FreezableMock __getattr__ needs a shortcut in case of return_value or side_effect
* Changing return_value only forbidden if set before freezing
* Remove unnecessary else block
* Expanded doc strings
* Bring a couple new tests in line with patch_get_utility() norms
* Add installer class
* Add wrapped reverter methods to common.Installer.
* Use Installer class in Apache plugin
* Use Installer class in Nginx plugin
* Don't create reverter in Apache and Nginx plugins
Remove unused help-related display code. When NcursesDisplay was
removed[1], help was deprecated. This change removes the remaining
bits and pieces of code.
Remove unused escape-related display code. When NcursesDisplay was
removed[1], escape was deprecated. This change removes the remaining
bits and pieces of code.
Remove uses of unused menu parameters.
Remove unused default_status/default_state argument from checklist.
(This seems safe because not only is it unused, the parameter has
different names in the interface and implementation)
1 - d54cb76432Resolves#4795.
When our Apache plugin is unable to determine which virtual host to use in
non-interactive mode, it raises an error about vhost ambiguity with
instructions on how to fix the problem. These instructions stated that we
require one vhost per file which is no longer accurate since #4706 so I removed
this part of the error message.
* move install_ssl_options_conf functionality to common
* add no cover
* compute current hash instead of saving
* make current hash be computed; switch to list of all canonical hashes
* put message directly into assertion
* don't pass logger
* add docstring
* Add unit tests for certbot.plugins.common.install_ssl_options_conf
* Revert "Pin python-augeas version to avoid error with 1.0.0 (#4422)"
This reverts commit 1c51ae2588.
* make dependency-requirements
* separate certbot and dependency requirements
* fix build.py
* update hashin comment
* simplify release pinning
* separate letsencrypt dependency
* pin hashes in venv
* error out when bad things happen
* use pinned dependencies in tox
* Revert "pin hashes in venv"
This reverts commit 1cd38a9e50.
* use pip_install.sh in venv_common
* quote pip install args
* bump mock version
* Case sensitivity fixes
* Clean up merge leftovers
* Get correct vhost paths when appending to already existing multivhost -le-ssl.conf
* Test, lint and reverter fixes
* Make py26 happy
* Removed skeletons
* Changed new vhost matching
* Added span flag for augeas init
* Extract VirtualHost using aug_span
* Removed dead code
* Fix tests to mitigate not being able to reload Augeas span values after write
* Small fixes and test coverage
* Implementing changes requested in review
* Don't filter vhosts on path if you've done so already
* add get_internal_aug_path
* Use relative augeas paths to determine if a file contains multiple virtual hosts
* get through parsing
* not slice
* add mult vhost per file
* idx line backwards
* blocks be wrong
* always close ifmod
* let's not mess up indexes
* don't double add multi
* fix some lint, only dedupe multi
* tests
* fix lint
* in progress bit flip
* try to pick the right vhost
* take Dominic's suggestion
* don't redo search
* add ancestor
* we now support multiple vhosts
* yay
* add docstrings
* Mention python 3 support in setup.py
* Build universal (py2 and py3 compatible) wheels
* Mention Python 3.3+ support in docs
* we work on python 3.6 too
When running ./tools/venv.sh with 1.0.0 (now the latest version), I encountered:
build/temp.linux-x86_64-2.7/augeas.c:434:35: fatal error: augeas.h: No such file or directory
* stop conditionally pinning mock version in acme
* stop conditionally pinning mock version in certbot
* stop conditionally pinning mock version in apache
* stop conditionally pinning mock version in nginx
* stop conditionally pinning mock version in letshelp
* stop conditionally pinning mock version in compatibility-test
Fixes#3996.
I'm pretty confident this PR solves the problem. I've audited all calls to IDisplay methods and the assertions done in certbot.display.util are now done in all our unit tests.
With that said, it wouldn't hurt to have someone else double check I didn't miss anything. The easiest way to do this is to grep for IDisplay in our code and ensure all calls to IDisplay methods are valid. This means every method call other than notification (because a notification call is always OK) either provides a value for default or force_interactive. This is defined in interfaces.py.
I've also been considering removing the assertion that's been causing us trouble here from our release. The only argument I have for not doing so is it may hinder 3rd party plugin development. When they use IDisplay, they have the same problem as we do with prompting users without a TTY. Not keeping this assertion in makes it more likely they won't notice the issue and Certbot will crash on an unsuspecting user.
With that said, none of our known 3rd party plugins use IDisplay at all.
* Provide force_interactive in _get_certname
* Use force_interactive when asking for webroot
* Factor IDisplay assertion into it's own function
* Add util.patch_get_utility()
* Allow custom path to patch_get_utiity
* Change GetEmailTest to use patch_get_utility
* Use new_callable to create new objects
* Modify tests to use patch_get_utility
* Improve FreezableMock documentation
* Add user facing error to TTY magic
* Comment out assert_valid_call
* Add test_input_assertion_fail2()
* correctly match * and ? in ServerAlias directives
* update Apache wildcard test
* Consolidate wildcard matching and remove bad test
* Test Apache vhost selection with wildcards
* Added few more tests to proof vhost selection
* CLI flag for forcing interactivity
* add --force-interactive
* Add force_interactive error checking and tests
* Add force_interactive parameter to FileDisplay
* add _can_interact
* Add _return_default
* Add **unused_kwargs to NoninteractiveDisplay
* improve _return_default assertion
* Change IDisplay calls and write tests
* Document force_interactive in interfaces.py
* Don't force_interactive with a new prompt
* Warn when skipping an interaction for the first time
* add specific logger.debug message
* Parallalelise nosetests from tox
* Parallelise even more things, break even more things
* Now unbreak all the tests that aren't ready for ||ism
* Try to pass tests!
- Remove non-working hack in reporter_test
- also be selective about ||ism in the cover environment
* Try again
* certbot-apache tests also work, given enough time
* Nginx may need more time in Travis's cloud
* Unbreak reporter_test under ||ism
* More timeout
* Working again?
* This goes way faster
* Another big win
* Split a couple more large test suites
* A last improvement
* More ||ism!
* ||ise lint too
* Allow nosetests to figure out how many cores to use
* simplify merge
* Mark the new CLI tests as ||izable
* Simplify reporter_test changes
* Rationalise ||ism flags
* Re-up coverage
* Clean up reporter tests
* Stop modifying testdata during tests
* remove unused os
* Begin breaking out cli_test.py
* simplify main
* refactor porse tests
* move determine account tests to main_test.py
* move duplicate cert test to main_test.py
* move cli stuff out of the way
* add test_renewal.py
* move error test into error_handler_test.py
* move test_read_file
* move test_no_gui out of MainTest
* move test_install_abspath to parsetest
* Move main tests into main_test.py
* move cli tests back into cli_test.py
* clean up cli_test.py
* move punycode test to util_test.py
* Fix NameError from missing plugins_disco
* Fix linting errors
* test_renewal.py -> renewal_test.py
* rm not_cli_test.py
* Move main._handle_exception test to main_test.py
* Move renewal import in renewal_test.py
from @ohemorange comments
* certbot.tests.test_util -> certbot.tests.util
* Fix issues from rebasing.
* Fix testing issue with option_was_set
* fix linting issue
* Allow running constants_test.py individually
* Mock until tests pass
Mock out both functions used to determine the OS in
certbot_apache.tests.constants_test.
