certbot/certbot-apache/certbot_apache/_internal/override_gentoo.py

76 lines
2.7 KiB
Python
Raw Permalink Normal View History

Distribution specific override functionality based on class inheritance (#5202) Class inheritance based approach to distro specific overrides. How it works: The certbot-apache plugin entrypoint has been changed to entrypoint.ENTRYPOINT which is a variable containing appropriate override class for system, if available. Override classes register themselves using decorator override.register() which takes a list of distribution fingerprints (ID & LIKE variables in /etc/os-release, or platform.linux_distribution() as a fallback). These end up as keys in dict override.OVERRIDE_CLASSES and values for the keys are references to the class that called the decorator, hence allowing self-registration of override classes when they are imported. The only file importing these override classes is entrypoint.py, so adding new override classes would need only one import in addition to the actual override class file. Generic changes: Parser initialization has been moved to separate class method, allowing easy override where needed. Cleaned up configurator.py a bit, and moved some helper functions to newly created apache_util.py Split Debian specific code from configurator.py to debian_override.py Changed define_cmd to apache_cmd because the parameters are for every distribution supporting this behavior, and we're able to use the value to build the additional configuration dump commands. Moved add_parser_mod() from configurator to parser add_mod() Added two new configuration dump parsing methods to update_runtime_variables() in parser: update_includes() and update_modules(). Changed init_modules() in parser to accommodate the changes above. (ie. don't throw existing self.modules out). Moved OS based constants to their respective override classes. Refactored configurator class discovery in tests to help easier test case creation using distribution based override configurator class. tests.util.get_apache_configurator() now takes keyword argument os_info which is string of the desired mock OS fingerprint response that's used for picking the right override class. This PR includes two major generic additions that should vastly improve our parsing accuracy and quality: Includes are parsed from config dump from httpd binary. This is mandatory for some distributions (Like OpenSUSE) to get visibility over the whole configuration tree because of Include statements passed on in command line, and not via root httpd.conf file. Modules are parsed from config dump from httpd binary. This lets us jump into correct IfModule directives if for some reason we have missed the module availability (because of one being included on command line or such). Distribution specific changes Because of the generic changes, there are two distributions (or distribution families) that do not provide such functionality, so it had to be overridden in their respective override files. These distributions are: CentOS, because it deliberately limits httpd binary stdout using SELinux as a feature. We are doing opportunistic config dumps here however, in case SELinux enforcing is off. Gentoo, because it does not provide a way to invoke httpd with command line parsed from its specific configuration file. Gentoo relies heavily on Define statements that are passed over from APACHE2_OPTS variable /etc/conf.d/apache2 file and most of the configuration in root Apache configuration are dependent on these values. Debian Moved the Debian specific parts from configurator.py to Debian specific override. CentOS Parsing of /etc/sysconfig/httpd file for additional Define statements. This could hold other parameters too, but parsing everything off it would require a full Apache lexer. For CLI parameters, I think Defines are the most common ones. This is done in addition of opportunistic parsing of httpd binary config dump. Added CentOS default Apache configuration tree for realistic test cases. Gentoo Parsing Defines from /etc/conf.d/apache2 variable APACHE2_OPTS, which holds additional Define statements to enable certain functionalities, enabling parts of the configuration in the Apache2 DOM. This is done instead of trying to parse httpd binary configuration dumps. Added default Apache configuration from Gentoo to testdata, including /etc/conf.d/apache2 file for realistic test cases. * Distribution specific override functionality based on class inheritance * Need to patch get_systemd_os_like to as travis has proper os-release * Added pydoc * Move parser initialization to a method and fix Python 3 __new__ errors * Parser changes to parse HTTPD config * Try to get modules and includes from httpd process for better visibility over the configuration * Had to disable duplicate-code because of test setup (PyCQA/pylint/issues/214) * CentOS tests and linter fixes * Gentoo override, tests and linter fixes * Mock the process call in all the tests that require it * Fix CentOS test mock * Restore reseting modules list functionality for cleanup * Move OS fingerprinting and constant mocks to parent class * Fixes requested in review * New entrypoint structure and started moving OS constants to override classes * OS constants move continued, test and linter fixes * Removed dead code * Apache compatibility test changest to reflect OS constant restructure * Test fix * Requested changes * Moved Debian specific tests to own test file * Removed decorator based override class registration in favor of entrypoint dict * Fix for update_includes for some versions of Augeas * Take fedora fix into account in tests * Review fixes
2017-12-04 14:49:18 -05:00
""" Distribution specific override class for Gentoo Linux """
Upgrade to mypy 0.812 (#8748) Fixes #8425 This PR upgrades mypy to the latest version available, 0.812. Given the advanced type inference capabilities provided by this newer version, this PRs also fixes various type inconsistencies that are now detected. Here are the non obvious changes done to fix types: * typing in mixins has been solved using `Protocol` classes, as recommended by mypy (https://mypy.readthedocs.io/en/latest/more_types.html#mixin-classes, https://mypy.readthedocs.io/en/stable/protocols.html) * `cast` when we are playing with `Union` types This PR also disables the strict optional checks that have been enable by default in recent versions of mypy. Once this PR is merged, I will create an issue to study how these checks can be enabled. `typing.Protocol` is available only since Python 3.8. To keep compatibility with Python 3.6, I try to import the class `Protocol` from `typing`, and fallback to assign `object` to `Protocol` if that fails. This way the code is working with all versions of Python, but the mypy check can be run only with Python 3.8+ because it needs the protocol feature. As a consequence, tox runs mypy under Python 3.8. Alternatives are: * importing `typing_extensions`, that proposes backport of newest typing features to Python 3.6, but this implies to add a dependency to Certbot just to run mypy * redesign the concerned classes to not use mixins, or use them differently, but this implies to modify the code itself even if there is nothing wrong with it and it is just a matter of instructing mypy to understand in which context the mixins can be used * ignoring type for these classes with `# type: ignore` but we loose the benefit of mypy for them * Upgrade mypy * First step for acme * Cast for the rescue * Fixing types for certbot * Fix typing for certbot-nginx * Finalize type fixes, configure no optional strict check for mypy in tox * Align requirements * Isort * Pylint * Protocol for python 3.6 * Use Python 3.9 for mypy, make code compatible with Python 3.8< * Pylint and mypy * Pragma no cover * Pythonic NotImplemented constant * More type definitions * Add comments * Simplify typing logic * Use vararg tuple * Relax constraints on mypy * Add more type * Do not silence error if target is not defined * Conditionally import Protocol for type checking only * Clean up imports * Add comments * Align python version linting with mypy and coverage * Just ignore types in an unused module * Add comments * Fix lint
2021-04-02 14:54:40 -04:00
from typing import cast
from typing import List
Distribution specific override functionality based on class inheritance (#5202) Class inheritance based approach to distro specific overrides. How it works: The certbot-apache plugin entrypoint has been changed to entrypoint.ENTRYPOINT which is a variable containing appropriate override class for system, if available. Override classes register themselves using decorator override.register() which takes a list of distribution fingerprints (ID & LIKE variables in /etc/os-release, or platform.linux_distribution() as a fallback). These end up as keys in dict override.OVERRIDE_CLASSES and values for the keys are references to the class that called the decorator, hence allowing self-registration of override classes when they are imported. The only file importing these override classes is entrypoint.py, so adding new override classes would need only one import in addition to the actual override class file. Generic changes: Parser initialization has been moved to separate class method, allowing easy override where needed. Cleaned up configurator.py a bit, and moved some helper functions to newly created apache_util.py Split Debian specific code from configurator.py to debian_override.py Changed define_cmd to apache_cmd because the parameters are for every distribution supporting this behavior, and we're able to use the value to build the additional configuration dump commands. Moved add_parser_mod() from configurator to parser add_mod() Added two new configuration dump parsing methods to update_runtime_variables() in parser: update_includes() and update_modules(). Changed init_modules() in parser to accommodate the changes above. (ie. don't throw existing self.modules out). Moved OS based constants to their respective override classes. Refactored configurator class discovery in tests to help easier test case creation using distribution based override configurator class. tests.util.get_apache_configurator() now takes keyword argument os_info which is string of the desired mock OS fingerprint response that's used for picking the right override class. This PR includes two major generic additions that should vastly improve our parsing accuracy and quality: Includes are parsed from config dump from httpd binary. This is mandatory for some distributions (Like OpenSUSE) to get visibility over the whole configuration tree because of Include statements passed on in command line, and not via root httpd.conf file. Modules are parsed from config dump from httpd binary. This lets us jump into correct IfModule directives if for some reason we have missed the module availability (because of one being included on command line or such). Distribution specific changes Because of the generic changes, there are two distributions (or distribution families) that do not provide such functionality, so it had to be overridden in their respective override files. These distributions are: CentOS, because it deliberately limits httpd binary stdout using SELinux as a feature. We are doing opportunistic config dumps here however, in case SELinux enforcing is off. Gentoo, because it does not provide a way to invoke httpd with command line parsed from its specific configuration file. Gentoo relies heavily on Define statements that are passed over from APACHE2_OPTS variable /etc/conf.d/apache2 file and most of the configuration in root Apache configuration are dependent on these values. Debian Moved the Debian specific parts from configurator.py to Debian specific override. CentOS Parsing of /etc/sysconfig/httpd file for additional Define statements. This could hold other parameters too, but parsing everything off it would require a full Apache lexer. For CLI parameters, I think Defines are the most common ones. This is done in addition of opportunistic parsing of httpd binary config dump. Added CentOS default Apache configuration tree for realistic test cases. Gentoo Parsing Defines from /etc/conf.d/apache2 variable APACHE2_OPTS, which holds additional Define statements to enable certain functionalities, enabling parts of the configuration in the Apache2 DOM. This is done instead of trying to parse httpd binary configuration dumps. Added default Apache configuration from Gentoo to testdata, including /etc/conf.d/apache2 file for realistic test cases. * Distribution specific override functionality based on class inheritance * Need to patch get_systemd_os_like to as travis has proper os-release * Added pydoc * Move parser initialization to a method and fix Python 3 __new__ errors * Parser changes to parse HTTPD config * Try to get modules and includes from httpd process for better visibility over the configuration * Had to disable duplicate-code because of test setup (PyCQA/pylint/issues/214) * CentOS tests and linter fixes * Gentoo override, tests and linter fixes * Mock the process call in all the tests that require it * Fix CentOS test mock * Restore reseting modules list functionality for cleanup * Move OS fingerprinting and constant mocks to parent class * Fixes requested in review * New entrypoint structure and started moving OS constants to override classes * OS constants move continued, test and linter fixes * Removed dead code * Apache compatibility test changest to reflect OS constant restructure * Test fix * Requested changes * Moved Debian specific tests to own test file * Removed decorator based override class registration in favor of entrypoint dict * Fix for update_includes for some versions of Augeas * Take fedora fix into account in tests * Review fixes
2017-12-04 14:49:18 -05:00
import zope.interface
from certbot import interfaces
Make the contents of the apache plugin private (#7579) Part of #5775. Tree: ``` certbot-apache/certbot_apache ├── __init__.py ├── _internal │   ├── apache_util.py │   ├── augeas_lens │   │   ├── httpd.aug │   │   └── README │   ├── centos-options-ssl-apache.conf │   ├── configurator.py │   ├── constants.py │   ├── display_ops.py │   ├── entrypoint.py │   ├── http_01.py │   ├── __init__.py │   ├── obj.py │   ├── options-ssl-apache.conf │   ├── override_arch.py │   ├── override_centos.py │   ├── override_darwin.py │   ├── override_debian.py │   ├── override_fedora.py │   ├── override_gentoo.py │   ├── override_suse.py │   └── parser.py └── tests ├── ... ``` * Create _internal folder for certbot_apache * Move apache_util.py to _internal * Move display_ops.py to _internal * Move override_centos.py to _internal * Move override_gentoo.py to _internal * Move override_darwin.py to _internal * Move override_suse.py to _internal * Move override_debian.py to _internal * Move override_fedora.py to _internal * Move override_arch.py to _internal * Move parser.py to _internal * Move obj.py to _internal * Move http_01.py to _internal * Move entrypoint.py to _internal * Move constants.py to _internal * Move configurator.py to _internal * Move augeas_lens to _internal * Move options-ssl-apache.conf files to _internal * move augeas_lens in MANIFEST * Clean up some stray references to certbot_apache that could use _internal * Correct imports and lint
2019-11-25 12:44:40 -05:00
from certbot_apache._internal import apache_util
from certbot_apache._internal import configurator
from certbot_apache._internal import parser
Distribution specific override functionality based on class inheritance (#5202) Class inheritance based approach to distro specific overrides. How it works: The certbot-apache plugin entrypoint has been changed to entrypoint.ENTRYPOINT which is a variable containing appropriate override class for system, if available. Override classes register themselves using decorator override.register() which takes a list of distribution fingerprints (ID & LIKE variables in /etc/os-release, or platform.linux_distribution() as a fallback). These end up as keys in dict override.OVERRIDE_CLASSES and values for the keys are references to the class that called the decorator, hence allowing self-registration of override classes when they are imported. The only file importing these override classes is entrypoint.py, so adding new override classes would need only one import in addition to the actual override class file. Generic changes: Parser initialization has been moved to separate class method, allowing easy override where needed. Cleaned up configurator.py a bit, and moved some helper functions to newly created apache_util.py Split Debian specific code from configurator.py to debian_override.py Changed define_cmd to apache_cmd because the parameters are for every distribution supporting this behavior, and we're able to use the value to build the additional configuration dump commands. Moved add_parser_mod() from configurator to parser add_mod() Added two new configuration dump parsing methods to update_runtime_variables() in parser: update_includes() and update_modules(). Changed init_modules() in parser to accommodate the changes above. (ie. don't throw existing self.modules out). Moved OS based constants to their respective override classes. Refactored configurator class discovery in tests to help easier test case creation using distribution based override configurator class. tests.util.get_apache_configurator() now takes keyword argument os_info which is string of the desired mock OS fingerprint response that's used for picking the right override class. This PR includes two major generic additions that should vastly improve our parsing accuracy and quality: Includes are parsed from config dump from httpd binary. This is mandatory for some distributions (Like OpenSUSE) to get visibility over the whole configuration tree because of Include statements passed on in command line, and not via root httpd.conf file. Modules are parsed from config dump from httpd binary. This lets us jump into correct IfModule directives if for some reason we have missed the module availability (because of one being included on command line or such). Distribution specific changes Because of the generic changes, there are two distributions (or distribution families) that do not provide such functionality, so it had to be overridden in their respective override files. These distributions are: CentOS, because it deliberately limits httpd binary stdout using SELinux as a feature. We are doing opportunistic config dumps here however, in case SELinux enforcing is off. Gentoo, because it does not provide a way to invoke httpd with command line parsed from its specific configuration file. Gentoo relies heavily on Define statements that are passed over from APACHE2_OPTS variable /etc/conf.d/apache2 file and most of the configuration in root Apache configuration are dependent on these values. Debian Moved the Debian specific parts from configurator.py to Debian specific override. CentOS Parsing of /etc/sysconfig/httpd file for additional Define statements. This could hold other parameters too, but parsing everything off it would require a full Apache lexer. For CLI parameters, I think Defines are the most common ones. This is done in addition of opportunistic parsing of httpd binary config dump. Added CentOS default Apache configuration tree for realistic test cases. Gentoo Parsing Defines from /etc/conf.d/apache2 variable APACHE2_OPTS, which holds additional Define statements to enable certain functionalities, enabling parts of the configuration in the Apache2 DOM. This is done instead of trying to parse httpd binary configuration dumps. Added default Apache configuration from Gentoo to testdata, including /etc/conf.d/apache2 file for realistic test cases. * Distribution specific override functionality based on class inheritance * Need to patch get_systemd_os_like to as travis has proper os-release * Added pydoc * Move parser initialization to a method and fix Python 3 __new__ errors * Parser changes to parse HTTPD config * Try to get modules and includes from httpd process for better visibility over the configuration * Had to disable duplicate-code because of test setup (PyCQA/pylint/issues/214) * CentOS tests and linter fixes * Gentoo override, tests and linter fixes * Mock the process call in all the tests that require it * Fix CentOS test mock * Restore reseting modules list functionality for cleanup * Move OS fingerprinting and constant mocks to parent class * Fixes requested in review * New entrypoint structure and started moving OS constants to override classes * OS constants move continued, test and linter fixes * Removed dead code * Apache compatibility test changest to reflect OS constant restructure * Test fix * Requested changes * Moved Debian specific tests to own test file * Removed decorator based override class registration in favor of entrypoint dict * Fix for update_includes for some versions of Augeas * Take fedora fix into account in tests * Review fixes
2017-12-04 14:49:18 -05:00
@zope.interface.provider(interfaces.IPluginFactory)
class GentooConfigurator(configurator.ApacheConfigurator):
"""Gentoo specific ApacheConfigurator override class"""
OS_DEFAULTS = dict(
server_root="/etc/apache2",
vhost_root="/etc/apache2/vhosts.d",
vhost_files="*.conf",
logs_root="/var/log/apache2",
ctl="apache2ctl",
version_cmd=['apache2ctl', '-v'],
Distribution specific override functionality based on class inheritance (#5202) Class inheritance based approach to distro specific overrides. How it works: The certbot-apache plugin entrypoint has been changed to entrypoint.ENTRYPOINT which is a variable containing appropriate override class for system, if available. Override classes register themselves using decorator override.register() which takes a list of distribution fingerprints (ID & LIKE variables in /etc/os-release, or platform.linux_distribution() as a fallback). These end up as keys in dict override.OVERRIDE_CLASSES and values for the keys are references to the class that called the decorator, hence allowing self-registration of override classes when they are imported. The only file importing these override classes is entrypoint.py, so adding new override classes would need only one import in addition to the actual override class file. Generic changes: Parser initialization has been moved to separate class method, allowing easy override where needed. Cleaned up configurator.py a bit, and moved some helper functions to newly created apache_util.py Split Debian specific code from configurator.py to debian_override.py Changed define_cmd to apache_cmd because the parameters are for every distribution supporting this behavior, and we're able to use the value to build the additional configuration dump commands. Moved add_parser_mod() from configurator to parser add_mod() Added two new configuration dump parsing methods to update_runtime_variables() in parser: update_includes() and update_modules(). Changed init_modules() in parser to accommodate the changes above. (ie. don't throw existing self.modules out). Moved OS based constants to their respective override classes. Refactored configurator class discovery in tests to help easier test case creation using distribution based override configurator class. tests.util.get_apache_configurator() now takes keyword argument os_info which is string of the desired mock OS fingerprint response that's used for picking the right override class. This PR includes two major generic additions that should vastly improve our parsing accuracy and quality: Includes are parsed from config dump from httpd binary. This is mandatory for some distributions (Like OpenSUSE) to get visibility over the whole configuration tree because of Include statements passed on in command line, and not via root httpd.conf file. Modules are parsed from config dump from httpd binary. This lets us jump into correct IfModule directives if for some reason we have missed the module availability (because of one being included on command line or such). Distribution specific changes Because of the generic changes, there are two distributions (or distribution families) that do not provide such functionality, so it had to be overridden in their respective override files. These distributions are: CentOS, because it deliberately limits httpd binary stdout using SELinux as a feature. We are doing opportunistic config dumps here however, in case SELinux enforcing is off. Gentoo, because it does not provide a way to invoke httpd with command line parsed from its specific configuration file. Gentoo relies heavily on Define statements that are passed over from APACHE2_OPTS variable /etc/conf.d/apache2 file and most of the configuration in root Apache configuration are dependent on these values. Debian Moved the Debian specific parts from configurator.py to Debian specific override. CentOS Parsing of /etc/sysconfig/httpd file for additional Define statements. This could hold other parameters too, but parsing everything off it would require a full Apache lexer. For CLI parameters, I think Defines are the most common ones. This is done in addition of opportunistic parsing of httpd binary config dump. Added CentOS default Apache configuration tree for realistic test cases. Gentoo Parsing Defines from /etc/conf.d/apache2 variable APACHE2_OPTS, which holds additional Define statements to enable certain functionalities, enabling parts of the configuration in the Apache2 DOM. This is done instead of trying to parse httpd binary configuration dumps. Added default Apache configuration from Gentoo to testdata, including /etc/conf.d/apache2 file for realistic test cases. * Distribution specific override functionality based on class inheritance * Need to patch get_systemd_os_like to as travis has proper os-release * Added pydoc * Move parser initialization to a method and fix Python 3 __new__ errors * Parser changes to parse HTTPD config * Try to get modules and includes from httpd process for better visibility over the configuration * Had to disable duplicate-code because of test setup (PyCQA/pylint/issues/214) * CentOS tests and linter fixes * Gentoo override, tests and linter fixes * Mock the process call in all the tests that require it * Fix CentOS test mock * Restore reseting modules list functionality for cleanup * Move OS fingerprinting and constant mocks to parent class * Fixes requested in review * New entrypoint structure and started moving OS constants to override classes * OS constants move continued, test and linter fixes * Removed dead code * Apache compatibility test changest to reflect OS constant restructure * Test fix * Requested changes * Moved Debian specific tests to own test file * Removed decorator based override class registration in favor of entrypoint dict * Fix for update_includes for some versions of Augeas * Take fedora fix into account in tests * Review fixes
2017-12-04 14:49:18 -05:00
restart_cmd=['apache2ctl', 'graceful'],
restart_cmd_alt=['apache2ctl', 'restart'],
Distribution specific override functionality based on class inheritance (#5202) Class inheritance based approach to distro specific overrides. How it works: The certbot-apache plugin entrypoint has been changed to entrypoint.ENTRYPOINT which is a variable containing appropriate override class for system, if available. Override classes register themselves using decorator override.register() which takes a list of distribution fingerprints (ID & LIKE variables in /etc/os-release, or platform.linux_distribution() as a fallback). These end up as keys in dict override.OVERRIDE_CLASSES and values for the keys are references to the class that called the decorator, hence allowing self-registration of override classes when they are imported. The only file importing these override classes is entrypoint.py, so adding new override classes would need only one import in addition to the actual override class file. Generic changes: Parser initialization has been moved to separate class method, allowing easy override where needed. Cleaned up configurator.py a bit, and moved some helper functions to newly created apache_util.py Split Debian specific code from configurator.py to debian_override.py Changed define_cmd to apache_cmd because the parameters are for every distribution supporting this behavior, and we're able to use the value to build the additional configuration dump commands. Moved add_parser_mod() from configurator to parser add_mod() Added two new configuration dump parsing methods to update_runtime_variables() in parser: update_includes() and update_modules(). Changed init_modules() in parser to accommodate the changes above. (ie. don't throw existing self.modules out). Moved OS based constants to their respective override classes. Refactored configurator class discovery in tests to help easier test case creation using distribution based override configurator class. tests.util.get_apache_configurator() now takes keyword argument os_info which is string of the desired mock OS fingerprint response that's used for picking the right override class. This PR includes two major generic additions that should vastly improve our parsing accuracy and quality: Includes are parsed from config dump from httpd binary. This is mandatory for some distributions (Like OpenSUSE) to get visibility over the whole configuration tree because of Include statements passed on in command line, and not via root httpd.conf file. Modules are parsed from config dump from httpd binary. This lets us jump into correct IfModule directives if for some reason we have missed the module availability (because of one being included on command line or such). Distribution specific changes Because of the generic changes, there are two distributions (or distribution families) that do not provide such functionality, so it had to be overridden in their respective override files. These distributions are: CentOS, because it deliberately limits httpd binary stdout using SELinux as a feature. We are doing opportunistic config dumps here however, in case SELinux enforcing is off. Gentoo, because it does not provide a way to invoke httpd with command line parsed from its specific configuration file. Gentoo relies heavily on Define statements that are passed over from APACHE2_OPTS variable /etc/conf.d/apache2 file and most of the configuration in root Apache configuration are dependent on these values. Debian Moved the Debian specific parts from configurator.py to Debian specific override. CentOS Parsing of /etc/sysconfig/httpd file for additional Define statements. This could hold other parameters too, but parsing everything off it would require a full Apache lexer. For CLI parameters, I think Defines are the most common ones. This is done in addition of opportunistic parsing of httpd binary config dump. Added CentOS default Apache configuration tree for realistic test cases. Gentoo Parsing Defines from /etc/conf.d/apache2 variable APACHE2_OPTS, which holds additional Define statements to enable certain functionalities, enabling parts of the configuration in the Apache2 DOM. This is done instead of trying to parse httpd binary configuration dumps. Added default Apache configuration from Gentoo to testdata, including /etc/conf.d/apache2 file for realistic test cases. * Distribution specific override functionality based on class inheritance * Need to patch get_systemd_os_like to as travis has proper os-release * Added pydoc * Move parser initialization to a method and fix Python 3 __new__ errors * Parser changes to parse HTTPD config * Try to get modules and includes from httpd process for better visibility over the configuration * Had to disable duplicate-code because of test setup (PyCQA/pylint/issues/214) * CentOS tests and linter fixes * Gentoo override, tests and linter fixes * Mock the process call in all the tests that require it * Fix CentOS test mock * Restore reseting modules list functionality for cleanup * Move OS fingerprinting and constant mocks to parent class * Fixes requested in review * New entrypoint structure and started moving OS constants to override classes * OS constants move continued, test and linter fixes * Removed dead code * Apache compatibility test changest to reflect OS constant restructure * Test fix * Requested changes * Moved Debian specific tests to own test file * Removed decorator based override class registration in favor of entrypoint dict * Fix for update_includes for some versions of Augeas * Take fedora fix into account in tests * Review fixes
2017-12-04 14:49:18 -05:00
conftest_cmd=['apache2ctl', 'configtest'],
enmod=None,
dismod=None,
le_vhost_ext="-le-ssl.conf",
handle_modules=False,
Distribution specific override functionality based on class inheritance (#5202) Class inheritance based approach to distro specific overrides. How it works: The certbot-apache plugin entrypoint has been changed to entrypoint.ENTRYPOINT which is a variable containing appropriate override class for system, if available. Override classes register themselves using decorator override.register() which takes a list of distribution fingerprints (ID & LIKE variables in /etc/os-release, or platform.linux_distribution() as a fallback). These end up as keys in dict override.OVERRIDE_CLASSES and values for the keys are references to the class that called the decorator, hence allowing self-registration of override classes when they are imported. The only file importing these override classes is entrypoint.py, so adding new override classes would need only one import in addition to the actual override class file. Generic changes: Parser initialization has been moved to separate class method, allowing easy override where needed. Cleaned up configurator.py a bit, and moved some helper functions to newly created apache_util.py Split Debian specific code from configurator.py to debian_override.py Changed define_cmd to apache_cmd because the parameters are for every distribution supporting this behavior, and we're able to use the value to build the additional configuration dump commands. Moved add_parser_mod() from configurator to parser add_mod() Added two new configuration dump parsing methods to update_runtime_variables() in parser: update_includes() and update_modules(). Changed init_modules() in parser to accommodate the changes above. (ie. don't throw existing self.modules out). Moved OS based constants to their respective override classes. Refactored configurator class discovery in tests to help easier test case creation using distribution based override configurator class. tests.util.get_apache_configurator() now takes keyword argument os_info which is string of the desired mock OS fingerprint response that's used for picking the right override class. This PR includes two major generic additions that should vastly improve our parsing accuracy and quality: Includes are parsed from config dump from httpd binary. This is mandatory for some distributions (Like OpenSUSE) to get visibility over the whole configuration tree because of Include statements passed on in command line, and not via root httpd.conf file. Modules are parsed from config dump from httpd binary. This lets us jump into correct IfModule directives if for some reason we have missed the module availability (because of one being included on command line or such). Distribution specific changes Because of the generic changes, there are two distributions (or distribution families) that do not provide such functionality, so it had to be overridden in their respective override files. These distributions are: CentOS, because it deliberately limits httpd binary stdout using SELinux as a feature. We are doing opportunistic config dumps here however, in case SELinux enforcing is off. Gentoo, because it does not provide a way to invoke httpd with command line parsed from its specific configuration file. Gentoo relies heavily on Define statements that are passed over from APACHE2_OPTS variable /etc/conf.d/apache2 file and most of the configuration in root Apache configuration are dependent on these values. Debian Moved the Debian specific parts from configurator.py to Debian specific override. CentOS Parsing of /etc/sysconfig/httpd file for additional Define statements. This could hold other parameters too, but parsing everything off it would require a full Apache lexer. For CLI parameters, I think Defines are the most common ones. This is done in addition of opportunistic parsing of httpd binary config dump. Added CentOS default Apache configuration tree for realistic test cases. Gentoo Parsing Defines from /etc/conf.d/apache2 variable APACHE2_OPTS, which holds additional Define statements to enable certain functionalities, enabling parts of the configuration in the Apache2 DOM. This is done instead of trying to parse httpd binary configuration dumps. Added default Apache configuration from Gentoo to testdata, including /etc/conf.d/apache2 file for realistic test cases. * Distribution specific override functionality based on class inheritance * Need to patch get_systemd_os_like to as travis has proper os-release * Added pydoc * Move parser initialization to a method and fix Python 3 __new__ errors * Parser changes to parse HTTPD config * Try to get modules and includes from httpd process for better visibility over the configuration * Had to disable duplicate-code because of test setup (PyCQA/pylint/issues/214) * CentOS tests and linter fixes * Gentoo override, tests and linter fixes * Mock the process call in all the tests that require it * Fix CentOS test mock * Restore reseting modules list functionality for cleanup * Move OS fingerprinting and constant mocks to parent class * Fixes requested in review * New entrypoint structure and started moving OS constants to override classes * OS constants move continued, test and linter fixes * Removed dead code * Apache compatibility test changest to reflect OS constant restructure * Test fix * Requested changes * Moved Debian specific tests to own test file * Removed decorator based override class registration in favor of entrypoint dict * Fix for update_includes for some versions of Augeas * Take fedora fix into account in tests * Review fixes
2017-12-04 14:49:18 -05:00
handle_sites=False,
challenge_location="/etc/apache2/vhosts.d",
bin=None,
Distribution specific override functionality based on class inheritance (#5202) Class inheritance based approach to distro specific overrides. How it works: The certbot-apache plugin entrypoint has been changed to entrypoint.ENTRYPOINT which is a variable containing appropriate override class for system, if available. Override classes register themselves using decorator override.register() which takes a list of distribution fingerprints (ID & LIKE variables in /etc/os-release, or platform.linux_distribution() as a fallback). These end up as keys in dict override.OVERRIDE_CLASSES and values for the keys are references to the class that called the decorator, hence allowing self-registration of override classes when they are imported. The only file importing these override classes is entrypoint.py, so adding new override classes would need only one import in addition to the actual override class file. Generic changes: Parser initialization has been moved to separate class method, allowing easy override where needed. Cleaned up configurator.py a bit, and moved some helper functions to newly created apache_util.py Split Debian specific code from configurator.py to debian_override.py Changed define_cmd to apache_cmd because the parameters are for every distribution supporting this behavior, and we're able to use the value to build the additional configuration dump commands. Moved add_parser_mod() from configurator to parser add_mod() Added two new configuration dump parsing methods to update_runtime_variables() in parser: update_includes() and update_modules(). Changed init_modules() in parser to accommodate the changes above. (ie. don't throw existing self.modules out). Moved OS based constants to their respective override classes. Refactored configurator class discovery in tests to help easier test case creation using distribution based override configurator class. tests.util.get_apache_configurator() now takes keyword argument os_info which is string of the desired mock OS fingerprint response that's used for picking the right override class. This PR includes two major generic additions that should vastly improve our parsing accuracy and quality: Includes are parsed from config dump from httpd binary. This is mandatory for some distributions (Like OpenSUSE) to get visibility over the whole configuration tree because of Include statements passed on in command line, and not via root httpd.conf file. Modules are parsed from config dump from httpd binary. This lets us jump into correct IfModule directives if for some reason we have missed the module availability (because of one being included on command line or such). Distribution specific changes Because of the generic changes, there are two distributions (or distribution families) that do not provide such functionality, so it had to be overridden in their respective override files. These distributions are: CentOS, because it deliberately limits httpd binary stdout using SELinux as a feature. We are doing opportunistic config dumps here however, in case SELinux enforcing is off. Gentoo, because it does not provide a way to invoke httpd with command line parsed from its specific configuration file. Gentoo relies heavily on Define statements that are passed over from APACHE2_OPTS variable /etc/conf.d/apache2 file and most of the configuration in root Apache configuration are dependent on these values. Debian Moved the Debian specific parts from configurator.py to Debian specific override. CentOS Parsing of /etc/sysconfig/httpd file for additional Define statements. This could hold other parameters too, but parsing everything off it would require a full Apache lexer. For CLI parameters, I think Defines are the most common ones. This is done in addition of opportunistic parsing of httpd binary config dump. Added CentOS default Apache configuration tree for realistic test cases. Gentoo Parsing Defines from /etc/conf.d/apache2 variable APACHE2_OPTS, which holds additional Define statements to enable certain functionalities, enabling parts of the configuration in the Apache2 DOM. This is done instead of trying to parse httpd binary configuration dumps. Added default Apache configuration from Gentoo to testdata, including /etc/conf.d/apache2 file for realistic test cases. * Distribution specific override functionality based on class inheritance * Need to patch get_systemd_os_like to as travis has proper os-release * Added pydoc * Move parser initialization to a method and fix Python 3 __new__ errors * Parser changes to parse HTTPD config * Try to get modules and includes from httpd process for better visibility over the configuration * Had to disable duplicate-code because of test setup (PyCQA/pylint/issues/214) * CentOS tests and linter fixes * Gentoo override, tests and linter fixes * Mock the process call in all the tests that require it * Fix CentOS test mock * Restore reseting modules list functionality for cleanup * Move OS fingerprinting and constant mocks to parent class * Fixes requested in review * New entrypoint structure and started moving OS constants to override classes * OS constants move continued, test and linter fixes * Removed dead code * Apache compatibility test changest to reflect OS constant restructure * Test fix * Requested changes * Moved Debian specific tests to own test file * Removed decorator based override class registration in favor of entrypoint dict * Fix for update_includes for some versions of Augeas * Take fedora fix into account in tests * Review fixes
2017-12-04 14:49:18 -05:00
)
def _prepare_options(self):
"""
Override the options dictionary initialization in order to support
alternative restart cmd used in Gentoo.
"""
super(GentooConfigurator, self)._prepare_options()
Upgrade to mypy 0.812 (#8748) Fixes #8425 This PR upgrades mypy to the latest version available, 0.812. Given the advanced type inference capabilities provided by this newer version, this PRs also fixes various type inconsistencies that are now detected. Here are the non obvious changes done to fix types: * typing in mixins has been solved using `Protocol` classes, as recommended by mypy (https://mypy.readthedocs.io/en/latest/more_types.html#mixin-classes, https://mypy.readthedocs.io/en/stable/protocols.html) * `cast` when we are playing with `Union` types This PR also disables the strict optional checks that have been enable by default in recent versions of mypy. Once this PR is merged, I will create an issue to study how these checks can be enabled. `typing.Protocol` is available only since Python 3.8. To keep compatibility with Python 3.6, I try to import the class `Protocol` from `typing`, and fallback to assign `object` to `Protocol` if that fails. This way the code is working with all versions of Python, but the mypy check can be run only with Python 3.8+ because it needs the protocol feature. As a consequence, tox runs mypy under Python 3.8. Alternatives are: * importing `typing_extensions`, that proposes backport of newest typing features to Python 3.6, but this implies to add a dependency to Certbot just to run mypy * redesign the concerned classes to not use mixins, or use them differently, but this implies to modify the code itself even if there is nothing wrong with it and it is just a matter of instructing mypy to understand in which context the mixins can be used * ignoring type for these classes with `# type: ignore` but we loose the benefit of mypy for them * Upgrade mypy * First step for acme * Cast for the rescue * Fixing types for certbot * Fix typing for certbot-nginx * Finalize type fixes, configure no optional strict check for mypy in tox * Align requirements * Isort * Pylint * Protocol for python 3.6 * Use Python 3.9 for mypy, make code compatible with Python 3.8< * Pylint and mypy * Pragma no cover * Pythonic NotImplemented constant * More type definitions * Add comments * Simplify typing logic * Use vararg tuple * Relax constraints on mypy * Add more type * Do not silence error if target is not defined * Conditionally import Protocol for type checking only * Clean up imports * Add comments * Align python version linting with mypy and coverage * Just ignore types in an unused module * Add comments * Fix lint
2021-04-02 14:54:40 -04:00
cast(List[str], self.options["restart_cmd_alt"])[0] = self.option("ctl")
Distribution specific override functionality based on class inheritance (#5202) Class inheritance based approach to distro specific overrides. How it works: The certbot-apache plugin entrypoint has been changed to entrypoint.ENTRYPOINT which is a variable containing appropriate override class for system, if available. Override classes register themselves using decorator override.register() which takes a list of distribution fingerprints (ID & LIKE variables in /etc/os-release, or platform.linux_distribution() as a fallback). These end up as keys in dict override.OVERRIDE_CLASSES and values for the keys are references to the class that called the decorator, hence allowing self-registration of override classes when they are imported. The only file importing these override classes is entrypoint.py, so adding new override classes would need only one import in addition to the actual override class file. Generic changes: Parser initialization has been moved to separate class method, allowing easy override where needed. Cleaned up configurator.py a bit, and moved some helper functions to newly created apache_util.py Split Debian specific code from configurator.py to debian_override.py Changed define_cmd to apache_cmd because the parameters are for every distribution supporting this behavior, and we're able to use the value to build the additional configuration dump commands. Moved add_parser_mod() from configurator to parser add_mod() Added two new configuration dump parsing methods to update_runtime_variables() in parser: update_includes() and update_modules(). Changed init_modules() in parser to accommodate the changes above. (ie. don't throw existing self.modules out). Moved OS based constants to their respective override classes. Refactored configurator class discovery in tests to help easier test case creation using distribution based override configurator class. tests.util.get_apache_configurator() now takes keyword argument os_info which is string of the desired mock OS fingerprint response that's used for picking the right override class. This PR includes two major generic additions that should vastly improve our parsing accuracy and quality: Includes are parsed from config dump from httpd binary. This is mandatory for some distributions (Like OpenSUSE) to get visibility over the whole configuration tree because of Include statements passed on in command line, and not via root httpd.conf file. Modules are parsed from config dump from httpd binary. This lets us jump into correct IfModule directives if for some reason we have missed the module availability (because of one being included on command line or such). Distribution specific changes Because of the generic changes, there are two distributions (or distribution families) that do not provide such functionality, so it had to be overridden in their respective override files. These distributions are: CentOS, because it deliberately limits httpd binary stdout using SELinux as a feature. We are doing opportunistic config dumps here however, in case SELinux enforcing is off. Gentoo, because it does not provide a way to invoke httpd with command line parsed from its specific configuration file. Gentoo relies heavily on Define statements that are passed over from APACHE2_OPTS variable /etc/conf.d/apache2 file and most of the configuration in root Apache configuration are dependent on these values. Debian Moved the Debian specific parts from configurator.py to Debian specific override. CentOS Parsing of /etc/sysconfig/httpd file for additional Define statements. This could hold other parameters too, but parsing everything off it would require a full Apache lexer. For CLI parameters, I think Defines are the most common ones. This is done in addition of opportunistic parsing of httpd binary config dump. Added CentOS default Apache configuration tree for realistic test cases. Gentoo Parsing Defines from /etc/conf.d/apache2 variable APACHE2_OPTS, which holds additional Define statements to enable certain functionalities, enabling parts of the configuration in the Apache2 DOM. This is done instead of trying to parse httpd binary configuration dumps. Added default Apache configuration from Gentoo to testdata, including /etc/conf.d/apache2 file for realistic test cases. * Distribution specific override functionality based on class inheritance * Need to patch get_systemd_os_like to as travis has proper os-release * Added pydoc * Move parser initialization to a method and fix Python 3 __new__ errors * Parser changes to parse HTTPD config * Try to get modules and includes from httpd process for better visibility over the configuration * Had to disable duplicate-code because of test setup (PyCQA/pylint/issues/214) * CentOS tests and linter fixes * Gentoo override, tests and linter fixes * Mock the process call in all the tests that require it * Fix CentOS test mock * Restore reseting modules list functionality for cleanup * Move OS fingerprinting and constant mocks to parent class * Fixes requested in review * New entrypoint structure and started moving OS constants to override classes * OS constants move continued, test and linter fixes * Removed dead code * Apache compatibility test changest to reflect OS constant restructure * Test fix * Requested changes * Moved Debian specific tests to own test file * Removed decorator based override class registration in favor of entrypoint dict * Fix for update_includes for some versions of Augeas * Take fedora fix into account in tests * Review fixes
2017-12-04 14:49:18 -05:00
def get_parser(self):
"""Initializes the ApacheParser"""
return GentooParser(
self.option("server_root"), self.option("vhost_root"),
Distribution specific override functionality based on class inheritance (#5202) Class inheritance based approach to distro specific overrides. How it works: The certbot-apache plugin entrypoint has been changed to entrypoint.ENTRYPOINT which is a variable containing appropriate override class for system, if available. Override classes register themselves using decorator override.register() which takes a list of distribution fingerprints (ID & LIKE variables in /etc/os-release, or platform.linux_distribution() as a fallback). These end up as keys in dict override.OVERRIDE_CLASSES and values for the keys are references to the class that called the decorator, hence allowing self-registration of override classes when they are imported. The only file importing these override classes is entrypoint.py, so adding new override classes would need only one import in addition to the actual override class file. Generic changes: Parser initialization has been moved to separate class method, allowing easy override where needed. Cleaned up configurator.py a bit, and moved some helper functions to newly created apache_util.py Split Debian specific code from configurator.py to debian_override.py Changed define_cmd to apache_cmd because the parameters are for every distribution supporting this behavior, and we're able to use the value to build the additional configuration dump commands. Moved add_parser_mod() from configurator to parser add_mod() Added two new configuration dump parsing methods to update_runtime_variables() in parser: update_includes() and update_modules(). Changed init_modules() in parser to accommodate the changes above. (ie. don't throw existing self.modules out). Moved OS based constants to their respective override classes. Refactored configurator class discovery in tests to help easier test case creation using distribution based override configurator class. tests.util.get_apache_configurator() now takes keyword argument os_info which is string of the desired mock OS fingerprint response that's used for picking the right override class. This PR includes two major generic additions that should vastly improve our parsing accuracy and quality: Includes are parsed from config dump from httpd binary. This is mandatory for some distributions (Like OpenSUSE) to get visibility over the whole configuration tree because of Include statements passed on in command line, and not via root httpd.conf file. Modules are parsed from config dump from httpd binary. This lets us jump into correct IfModule directives if for some reason we have missed the module availability (because of one being included on command line or such). Distribution specific changes Because of the generic changes, there are two distributions (or distribution families) that do not provide such functionality, so it had to be overridden in their respective override files. These distributions are: CentOS, because it deliberately limits httpd binary stdout using SELinux as a feature. We are doing opportunistic config dumps here however, in case SELinux enforcing is off. Gentoo, because it does not provide a way to invoke httpd with command line parsed from its specific configuration file. Gentoo relies heavily on Define statements that are passed over from APACHE2_OPTS variable /etc/conf.d/apache2 file and most of the configuration in root Apache configuration are dependent on these values. Debian Moved the Debian specific parts from configurator.py to Debian specific override. CentOS Parsing of /etc/sysconfig/httpd file for additional Define statements. This could hold other parameters too, but parsing everything off it would require a full Apache lexer. For CLI parameters, I think Defines are the most common ones. This is done in addition of opportunistic parsing of httpd binary config dump. Added CentOS default Apache configuration tree for realistic test cases. Gentoo Parsing Defines from /etc/conf.d/apache2 variable APACHE2_OPTS, which holds additional Define statements to enable certain functionalities, enabling parts of the configuration in the Apache2 DOM. This is done instead of trying to parse httpd binary configuration dumps. Added default Apache configuration from Gentoo to testdata, including /etc/conf.d/apache2 file for realistic test cases. * Distribution specific override functionality based on class inheritance * Need to patch get_systemd_os_like to as travis has proper os-release * Added pydoc * Move parser initialization to a method and fix Python 3 __new__ errors * Parser changes to parse HTTPD config * Try to get modules and includes from httpd process for better visibility over the configuration * Had to disable duplicate-code because of test setup (PyCQA/pylint/issues/214) * CentOS tests and linter fixes * Gentoo override, tests and linter fixes * Mock the process call in all the tests that require it * Fix CentOS test mock * Restore reseting modules list functionality for cleanup * Move OS fingerprinting and constant mocks to parent class * Fixes requested in review * New entrypoint structure and started moving OS constants to override classes * OS constants move continued, test and linter fixes * Removed dead code * Apache compatibility test changest to reflect OS constant restructure * Test fix * Requested changes * Moved Debian specific tests to own test file * Removed decorator based override class registration in favor of entrypoint dict * Fix for update_includes for some versions of Augeas * Take fedora fix into account in tests * Review fixes
2017-12-04 14:49:18 -05:00
self.version, configurator=self)
class GentooParser(parser.ApacheParser):
"""Gentoo specific ApacheParser override class"""
def __init__(self, *args, **kwargs):
# Gentoo specific configuration file for Apache2
self.apacheconfig_filep = "/etc/conf.d/apache2"
super(GentooParser, self).__init__(*args, **kwargs)
def update_runtime_variables(self):
""" Override for update_runtime_variables for custom parsing """
self.parse_sysconfig_var()
self.update_modules()
Distribution specific override functionality based on class inheritance (#5202) Class inheritance based approach to distro specific overrides. How it works: The certbot-apache plugin entrypoint has been changed to entrypoint.ENTRYPOINT which is a variable containing appropriate override class for system, if available. Override classes register themselves using decorator override.register() which takes a list of distribution fingerprints (ID & LIKE variables in /etc/os-release, or platform.linux_distribution() as a fallback). These end up as keys in dict override.OVERRIDE_CLASSES and values for the keys are references to the class that called the decorator, hence allowing self-registration of override classes when they are imported. The only file importing these override classes is entrypoint.py, so adding new override classes would need only one import in addition to the actual override class file. Generic changes: Parser initialization has been moved to separate class method, allowing easy override where needed. Cleaned up configurator.py a bit, and moved some helper functions to newly created apache_util.py Split Debian specific code from configurator.py to debian_override.py Changed define_cmd to apache_cmd because the parameters are for every distribution supporting this behavior, and we're able to use the value to build the additional configuration dump commands. Moved add_parser_mod() from configurator to parser add_mod() Added two new configuration dump parsing methods to update_runtime_variables() in parser: update_includes() and update_modules(). Changed init_modules() in parser to accommodate the changes above. (ie. don't throw existing self.modules out). Moved OS based constants to their respective override classes. Refactored configurator class discovery in tests to help easier test case creation using distribution based override configurator class. tests.util.get_apache_configurator() now takes keyword argument os_info which is string of the desired mock OS fingerprint response that's used for picking the right override class. This PR includes two major generic additions that should vastly improve our parsing accuracy and quality: Includes are parsed from config dump from httpd binary. This is mandatory for some distributions (Like OpenSUSE) to get visibility over the whole configuration tree because of Include statements passed on in command line, and not via root httpd.conf file. Modules are parsed from config dump from httpd binary. This lets us jump into correct IfModule directives if for some reason we have missed the module availability (because of one being included on command line or such). Distribution specific changes Because of the generic changes, there are two distributions (or distribution families) that do not provide such functionality, so it had to be overridden in their respective override files. These distributions are: CentOS, because it deliberately limits httpd binary stdout using SELinux as a feature. We are doing opportunistic config dumps here however, in case SELinux enforcing is off. Gentoo, because it does not provide a way to invoke httpd with command line parsed from its specific configuration file. Gentoo relies heavily on Define statements that are passed over from APACHE2_OPTS variable /etc/conf.d/apache2 file and most of the configuration in root Apache configuration are dependent on these values. Debian Moved the Debian specific parts from configurator.py to Debian specific override. CentOS Parsing of /etc/sysconfig/httpd file for additional Define statements. This could hold other parameters too, but parsing everything off it would require a full Apache lexer. For CLI parameters, I think Defines are the most common ones. This is done in addition of opportunistic parsing of httpd binary config dump. Added CentOS default Apache configuration tree for realistic test cases. Gentoo Parsing Defines from /etc/conf.d/apache2 variable APACHE2_OPTS, which holds additional Define statements to enable certain functionalities, enabling parts of the configuration in the Apache2 DOM. This is done instead of trying to parse httpd binary configuration dumps. Added default Apache configuration from Gentoo to testdata, including /etc/conf.d/apache2 file for realistic test cases. * Distribution specific override functionality based on class inheritance * Need to patch get_systemd_os_like to as travis has proper os-release * Added pydoc * Move parser initialization to a method and fix Python 3 __new__ errors * Parser changes to parse HTTPD config * Try to get modules and includes from httpd process for better visibility over the configuration * Had to disable duplicate-code because of test setup (PyCQA/pylint/issues/214) * CentOS tests and linter fixes * Gentoo override, tests and linter fixes * Mock the process call in all the tests that require it * Fix CentOS test mock * Restore reseting modules list functionality for cleanup * Move OS fingerprinting and constant mocks to parent class * Fixes requested in review * New entrypoint structure and started moving OS constants to override classes * OS constants move continued, test and linter fixes * Removed dead code * Apache compatibility test changest to reflect OS constant restructure * Test fix * Requested changes * Moved Debian specific tests to own test file * Removed decorator based override class registration in favor of entrypoint dict * Fix for update_includes for some versions of Augeas * Take fedora fix into account in tests * Review fixes
2017-12-04 14:49:18 -05:00
def parse_sysconfig_var(self):
""" Parses Apache CLI options from Gentoo configuration file """
defines = apache_util.parse_define_file(self.apacheconfig_filep,
"APACHE2_OPTS")
for k in defines:
Distribution specific override functionality based on class inheritance (#5202) Class inheritance based approach to distro specific overrides. How it works: The certbot-apache plugin entrypoint has been changed to entrypoint.ENTRYPOINT which is a variable containing appropriate override class for system, if available. Override classes register themselves using decorator override.register() which takes a list of distribution fingerprints (ID & LIKE variables in /etc/os-release, or platform.linux_distribution() as a fallback). These end up as keys in dict override.OVERRIDE_CLASSES and values for the keys are references to the class that called the decorator, hence allowing self-registration of override classes when they are imported. The only file importing these override classes is entrypoint.py, so adding new override classes would need only one import in addition to the actual override class file. Generic changes: Parser initialization has been moved to separate class method, allowing easy override where needed. Cleaned up configurator.py a bit, and moved some helper functions to newly created apache_util.py Split Debian specific code from configurator.py to debian_override.py Changed define_cmd to apache_cmd because the parameters are for every distribution supporting this behavior, and we're able to use the value to build the additional configuration dump commands. Moved add_parser_mod() from configurator to parser add_mod() Added two new configuration dump parsing methods to update_runtime_variables() in parser: update_includes() and update_modules(). Changed init_modules() in parser to accommodate the changes above. (ie. don't throw existing self.modules out). Moved OS based constants to their respective override classes. Refactored configurator class discovery in tests to help easier test case creation using distribution based override configurator class. tests.util.get_apache_configurator() now takes keyword argument os_info which is string of the desired mock OS fingerprint response that's used for picking the right override class. This PR includes two major generic additions that should vastly improve our parsing accuracy and quality: Includes are parsed from config dump from httpd binary. This is mandatory for some distributions (Like OpenSUSE) to get visibility over the whole configuration tree because of Include statements passed on in command line, and not via root httpd.conf file. Modules are parsed from config dump from httpd binary. This lets us jump into correct IfModule directives if for some reason we have missed the module availability (because of one being included on command line or such). Distribution specific changes Because of the generic changes, there are two distributions (or distribution families) that do not provide such functionality, so it had to be overridden in their respective override files. These distributions are: CentOS, because it deliberately limits httpd binary stdout using SELinux as a feature. We are doing opportunistic config dumps here however, in case SELinux enforcing is off. Gentoo, because it does not provide a way to invoke httpd with command line parsed from its specific configuration file. Gentoo relies heavily on Define statements that are passed over from APACHE2_OPTS variable /etc/conf.d/apache2 file and most of the configuration in root Apache configuration are dependent on these values. Debian Moved the Debian specific parts from configurator.py to Debian specific override. CentOS Parsing of /etc/sysconfig/httpd file for additional Define statements. This could hold other parameters too, but parsing everything off it would require a full Apache lexer. For CLI parameters, I think Defines are the most common ones. This is done in addition of opportunistic parsing of httpd binary config dump. Added CentOS default Apache configuration tree for realistic test cases. Gentoo Parsing Defines from /etc/conf.d/apache2 variable APACHE2_OPTS, which holds additional Define statements to enable certain functionalities, enabling parts of the configuration in the Apache2 DOM. This is done instead of trying to parse httpd binary configuration dumps. Added default Apache configuration from Gentoo to testdata, including /etc/conf.d/apache2 file for realistic test cases. * Distribution specific override functionality based on class inheritance * Need to patch get_systemd_os_like to as travis has proper os-release * Added pydoc * Move parser initialization to a method and fix Python 3 __new__ errors * Parser changes to parse HTTPD config * Try to get modules and includes from httpd process for better visibility over the configuration * Had to disable duplicate-code because of test setup (PyCQA/pylint/issues/214) * CentOS tests and linter fixes * Gentoo override, tests and linter fixes * Mock the process call in all the tests that require it * Fix CentOS test mock * Restore reseting modules list functionality for cleanup * Move OS fingerprinting and constant mocks to parent class * Fixes requested in review * New entrypoint structure and started moving OS constants to override classes * OS constants move continued, test and linter fixes * Removed dead code * Apache compatibility test changest to reflect OS constant restructure * Test fix * Requested changes * Moved Debian specific tests to own test file * Removed decorator based override class registration in favor of entrypoint dict * Fix for update_includes for some versions of Augeas * Take fedora fix into account in tests * Review fixes
2017-12-04 14:49:18 -05:00
self.variables[k] = defines[k]
def update_modules(self):
"""Get loaded modules from httpd process, and add them to DOM"""
mod_cmd = [self.configurator.option("ctl"), "modules"]
matches = apache_util.parse_from_subprocess(mod_cmd, r"(.*)_module")
for mod in matches:
self.add_mod(mod.strip())