upstream/certbot
1
0
Fork 0
mirror of https://github.com/certbot/certbot.git synced 2026-06-05 06:42:10 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
certbot/tox.ini

292 lines
9.9 KiB
INI
Raw Permalink Normal View History

Convert http links to https (#8287) * Convert http links to https * Fix remaining links
2020-09-23 13:36:55 -04:00
# Tox (https://tox.readthedocs.io/) is a tool for running tests in
Simpler Travis CI matrix
2015-01-28 07:56:12 -05:00
# multiple virtualenvs. To use it, "pip install tox" and then run
# "tox" from this directory.
Use tox for tests
2014-11-22 09:02:52 -05:00
[tox]
Basic dev/test setup for separate package subdirectories.
2015-07-10 12:34:08 -04:00
skipsdist = true
Consolidate cover envs and default to py3-cover (#7905) * Consolidate cover envs and default to py3-cover * use py38 for code coverage in Travis * Disable coverage on Python < 3.6 line.
2020-04-16 11:59:40 -04:00
envlist = modification,py3-cover,lint,mypy
Revert switch to `python setup.py test` in tox.ini. This had more of a purpose when we were moving mock to test_requires. I'll reintroduce this in the separate PR for that. Also bring back the testing extra in tox for now.
2016-01-19 12:22:53 -05:00
Deduplicate package lists in tox.ini (#4608) Use substitution of values form other sections[1] to deduplicate information in tox.ini, including pip install arguments and package paths. 1 - https://tox.readthedocs.io/en/latest/config.html#substitution-for-values-from-other-sections
2017-05-08 14:54:12 -04:00
[base]
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
# pip installs the requested packages in editable mode
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
pip_install = python {toxinidir}/tools/pip_install_editable.py
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
# pip installs the requested packages in editable mode and runs unit tests on
# them. Each package is installed and tested in the order they are provided
Pin dependencies in oldest tests (#5316) * Add tools/merge_requirements.py * Revert "Fix oldest tests by pinning Google DNS deps (#5000)" This reverts commit f68fba2be2fc342dd72deaaf048ab79e5a8fc2be. * Add tools/oldest_constraints.txt * Remove oldest constraints from tox.ini * Rename dev constraints file * Update tools/pip_install.sh * Update install_and_test.sh * Fix pip_install.sh * Don't cat when you can cp * Add ng-httpsclient to dev constraints for oldest tests * Bump tested setuptools version * Update dev_constraints comment * Better document oldest dependencies * test against oldest versions we say we require * Update dev constraints * Properly handle empty lines * Update constraints gen in pip_install * Remove duplicated zope.component * Reduce pyasn1-modules dependency * Remove blank line * pin back google-api-python-client * pin back uritemplate * pin josepy for oldest tests * Undo changes to install_and_test.sh * Update install_and_test.sh description * use split instead of partition
2017-12-18 15:31:36 -05:00
# before the script moves on to the next package. All dependencies are pinned
# to a specific version for increased stability for developers.
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
install_and_test = python {toxinidir}/tools/install_and_test.py
Full Python 3.7 support (#6182) Now that yaml/pyyaml#126 is resolved, #6170 can be reverted by bumping the pinned version of PyYAML. You can see this code passing with full macOS and integration tests at https://travis-ci.org/certbot/certbot/builds/400957729. * Revert "Allow py37 testing (#6170)" This reverts commit cad95466b05e6be51c1c29eaa91e6e3b7ea3cefd. * Bump pyyaml pinning to work on Python 3.7.
2018-07-09 12:16:44 -04:00
dns_packages =
certbot-dns-cloudflare \
Drop support for EOL Python 2.6 and 3.3 * Drop support for EOL Python 2.6 * Use more helpful assertIn/NotIn instead of assertTrue/False * Drop support for EOL Python 3.3 * Remove redundant Python 3.3 code * Restore code for RHEL 6 and virtualenv for Py2.7 * Revert pipstrap.py to upstream * Merge py26_packages and non_py26_packages into all_packages * Revert changes to *-auto in root * Update by calling letsencrypt-auto-source/build.py * Revert permissions for pipstrap.py
2018-01-26 04:47:25 -05:00
certbot-dns-cloudxns \
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
certbot-dns-digitalocean \
Drop support for EOL Python 2.6 and 3.3 * Drop support for EOL Python 2.6 * Use more helpful assertIn/NotIn instead of assertTrue/False * Drop support for EOL Python 3.3 * Remove redundant Python 3.3 code * Restore code for RHEL 6 and virtualenv for Py2.7 * Revert pipstrap.py to upstream * Merge py26_packages and non_py26_packages into all_packages * Revert changes to *-auto in root * Update by calling letsencrypt-auto-source/build.py * Revert permissions for pipstrap.py
2018-01-26 04:47:25 -05:00
certbot-dns-dnsimple \
certbot-dns-dnsmadeeasy \
Gehirn Infrastracture Service DNS Authenticator (#5702) Implement an Authenticator which can fulfill a dns-01 challenge using the Gehirn DNS (Gehirn Infrastructure Service) API. Applicable only for domains using Gehirn DNS for DNS. Testing Done: * `tox -e py27` * `tox -e lint` * Manual testing: * Used `certbot certonly --dns-gehirn -d`, specifying a credentials file as a command line argument. Verified that a certificate was successfully obtained without user interaction. * Negative testing: * Path to non-existent credentials file. * Credentials file with unsafe permissions (644). * Domain name not registered to Gehirn DNS account.
2018-07-10 20:36:20 -04:00
certbot-dns-gehirn \
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
certbot-dns-google \
Linode DNS Authenticator (#5302) * Added DNS based authenticator plugin for Linode * Added linode plugin to docs * Added Dockerfile * Added .gitignore and readthedocs.org.requirements.txt * Updated default_propagation_seconds * Updated according to changes requested * Bump version to 0.26.0 * Advertise our packages work on Python 3.7.
2018-07-10 16:51:03 -04:00
certbot-dns-linode \
Drop support for EOL Python 2.6 and 3.3 * Drop support for EOL Python 2.6 * Use more helpful assertIn/NotIn instead of assertTrue/False * Drop support for EOL Python 3.3 * Remove redundant Python 3.3 code * Restore code for RHEL 6 and virtualenv for Py2.7 * Revert pipstrap.py to upstream * Merge py26_packages and non_py26_packages into all_packages * Revert changes to *-auto in root * Update by calling letsencrypt-auto-source/build.py * Revert permissions for pipstrap.py
2018-01-26 04:47:25 -05:00
certbot-dns-luadns \
certbot-dns-nsone \
OVH DNS Authenticator (#5423) Implement an Authenticator which can fulfill a dns-01 challenge using the OVH DNS API. Applicable only for domains using OVH DNS. Testing Done: * `tox -e py27` * `tox -e lint` * Manual testing: * Used `certbot certonly --dns-ovh -d`, specifying a credentials file as a command line argument. Verified that a certificate was successfully obtained without user interaction. * Used `certbot certonly --dns-ovh -d`, without specifying a credentials file as a command line argument. Verified that the user was prompted and that a certificate was successfully obtained. * Used `certbot certonly -d`. Verified that the user was prompted for a credentials file after selecting dnsimple interactively and that a certificate was successfully obtained. * Used `certbot renew --force-renewal`. Verified that certificates were renewed without user interaction. * Negative testing: * Path to non-existent credentials file. * Credentials file with unsafe permissions (644). * Path to credentials file with an invalid application key. * Path to credentials file with an invalid application secret. * Path to credentials file with an invalid consumer key. * Path to credentials file with missing properties. * Domain name not registered to OVH account.
2018-07-10 23:52:32 -04:00
certbot-dns-ovh \
RFC 2136 DNS Authenticator (#4701) Introduce a plugin that automates the process of completing a dns-01 challenge by creating, and subsequently removing, TXT records using RFC 2136 Dynamic Updates (a.k.a. nsupdate). This plugin has been tested with BIND, but may work with other RFC 2136-compatible DNS servers, such as PowerDNS.
2017-06-16 19:53:46 -04:00
certbot-dns-rfc2136 \
Sakura Cloud DNS Authenticator (#5701) Implement an Authenticator which can fulfill a dns-01 challenge using the Sakura Cloud DNS API. Applicable only for domains using Sakura Cloud for DNS. Testing Done: * `tox -e py27` * `tox -e lint` * Manual testing: * Used `certbot certonly --dns-sakuracloud -d`, specifying a credentials file as a command line argument. Verified that a certificate was successfully obtained without user interaction. * Negative testing: * Path to non-existent credentials file. * Credentials file with unsafe permissions (644). * Domain name not registered to Sakura Cloud account.
2018-07-10 17:30:37 -04:00
certbot-dns-route53 \
certbot-dns-sakuracloud
Full Python 3.7 support (#6182) Now that yaml/pyyaml#126 is resolved, #6170 can be reverted by bumping the pinned version of PyYAML. You can see this code passing with full macOS and integration tests at https://travis-ci.org/certbot/certbot/builds/400957729. * Revert "Allow py37 testing (#6170)" This reverts commit cad95466b05e6be51c1c29eaa91e6e3b7ea3cefd. * Bump pyyaml pinning to work on Python 3.7.
2018-07-09 12:16:44 -04:00
all_packages =
Break lockstep between our packages (#5655) Fixes #5490. There's a lot of possibilities discussed in #5490, but I'll try and explain what I actually did here as succinctly as I can. Unfortunately, there's a fair bit to explain. My goal was to break lockstep and give us tests to ensure the minimum specified versions are correct without taking the time now to refactor our whole test setup. To handle specifying each package's minimum acme/certbot version, I added a requirements file to each package. This won't actually be included in the shipped package (because it's not in the MANIFEST). After creating these files and modifying tools/pip_install.sh to use them, I created a separate tox env for most packages (I kept the DNS plugins together for convenience). The reason this is necessary is because we currently use a single environment for each plugin, but if we used this approach for these tests we'd hit issues due to different installed plugins requiring different versions of acme/certbot. There's a lot more discussion about this in #5490 if you're interested in this piece. I unfortunately wasted a lot of time trying to remove the boilerplate this approach causes in tox.ini, but to do this I think we need negations described at complex factor conditions which hasn't made it into a tox release yet. The biggest missing piece here is how to make sure the oldest versions that are currently pinned to master get updated. Currently, they'll stay pinned that way without manual intervention and won't be properly testing the oldest version. I think we should solve this during the larger test/repo refactoring after the release because the tests are using the correct values now and I don't see a simple way around the problem. Once this lands, I'm planning on updating the test-everything tests to do integration tests with the "oldest" versions here. * break lockstep between packages * Use per package requirements files * add local oldest requirements files * update tox.ini * work with dev0 versions * Install requirements in separate step. * don't error when we don't have requirements * install latest packages in editable mode * Update .travis.yml * Add reminder comments * move dev to requirements * request acme[dev] * Update pip_install documentation
2018-03-05 12:50:19 -05:00
acme[dev] \
Refactor certbot/ and certbot/tests/ to use the same structure as the other packages (#7544) Summary of changes in this PR: - Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below). - Make repo root README symlink to `certbot` README. - Pull tests outside of the distributed module. - Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery. - Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments. - Replace mentions of `.` with `certbot` when referring to packages to install, usually editably. - Clean up some unused code around executing tests in a different directory. - Create public shim around main and make that the entry point. New directory structure summary: ``` repo root ("certbot", probably, but for clarity all files I mention are relative to here) ├── certbot │   ├── setup.py │   ├── certbot │   │   ├── __init__.py │   │   ├── achallenges.py │   │   ├── _internal │   │   │   ├── __init__.py │   │   │   ├── account.py │   │   │   ├── ... │   │   ├── ... │   ├── tests │   │   ├── account_test.py │   │   ├── display │   │   │   ├── __init__.py │   │   │   ├── ... │   │   ├── ... # note no __init__.py at this level │   ├── ... ├── acme │   ├── ... ├── certbot-apache │   ├── ... ├── ... ``` * refactor certbot/ and certbot/tests/ to use the same structure as the other packages * git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g" * git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g" * git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g" * Remove replacement of certbot into . in install_and_test.py * copy license back out to main folder * remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/ * Move README back into main folder, and make the version inside certbot/ a symlink * symlink certbot READMEs the other way around * move testdata into the public api certbot zone * update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests * certbot version has been bumped down a directory level * make certbot tests directory not a package and import sibling as module * Remove unused script cruft * change . to certbot in test_sdists * remove outdated comment referencing a command that doesn't work * Install instructions should reference an existing file * update file paths in Dockerfile * some package named in tox.ini were manually specified, change those to certbot * new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code * remove other instance of pyargs * fix up some references in _release.sh by searching for ' . ' and manual check * another stray . in tox.ini * fix paths in tools/_release.sh * Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper * Create public shim around main and make that the entry point * without pyargs, tests cannot be run from an empty directory * Remove cruft for running certbot directly from main * Have main shim take real arg * add docs/api file for main, and fix up main comment * Update certbot/docs/install.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Fix comments in readthedocs requirements files to refer to current package * Update .[docs] reference in contributing.rst * Move plugins tests to certbot tests directory * add certbot tests to MANIFEST.in so packagers can run python setup.py test * move examples directory inside certbot/ * Move CHANGELOG into certbot, and create a top-level symlink * Remove unused sys and logging from main shim * nginx http01 test no longer relies on certbot plugins common test
2019-11-25 17:28:06 -05:00
certbot[dev] \
Break lockstep between our packages (#5655) Fixes #5490. There's a lot of possibilities discussed in #5490, but I'll try and explain what I actually did here as succinctly as I can. Unfortunately, there's a fair bit to explain. My goal was to break lockstep and give us tests to ensure the minimum specified versions are correct without taking the time now to refactor our whole test setup. To handle specifying each package's minimum acme/certbot version, I added a requirements file to each package. This won't actually be included in the shipped package (because it's not in the MANIFEST). After creating these files and modifying tools/pip_install.sh to use them, I created a separate tox env for most packages (I kept the DNS plugins together for convenience). The reason this is necessary is because we currently use a single environment for each plugin, but if we used this approach for these tests we'd hit issues due to different installed plugins requiring different versions of acme/certbot. There's a lot more discussion about this in #5490 if you're interested in this piece. I unfortunately wasted a lot of time trying to remove the boilerplate this approach causes in tox.ini, but to do this I think we need negations described at complex factor conditions which hasn't made it into a tox release yet. The biggest missing piece here is how to make sure the oldest versions that are currently pinned to master get updated. Currently, they'll stay pinned that way without manual intervention and won't be properly testing the oldest version. I think we should solve this during the larger test/repo refactoring after the release because the tests are using the correct values now and I don't see a simple way around the problem. Once this lands, I'm planning on updating the test-everything tests to do integration tests with the "oldest" versions here. * break lockstep between packages * Use per package requirements files * add local oldest requirements files * update tox.ini * work with dev0 versions * Install requirements in separate step. * don't error when we don't have requirements * install latest packages in editable mode * Update .travis.yml * Add reminder comments * move dev to requirements * request acme[dev] * Update pip_install documentation
2018-03-05 12:50:19 -05:00
certbot-apache \
Full Python 3.7 support (#6182) Now that yaml/pyyaml#126 is resolved, #6170 can be reverted by bumping the pinned version of PyYAML. You can see this code passing with full macOS and integration tests at https://travis-ci.org/certbot/certbot/builds/400957729. * Revert "Allow py37 testing (#6170)" This reverts commit cad95466b05e6be51c1c29eaa91e6e3b7ea3cefd. * Bump pyyaml pinning to work on Python 3.7.
2018-07-09 12:16:44 -04:00
{[base]dns_packages} \
Remove letshelp-certbot (#7761) * remove references to letshelp * remove letshelp files * Remove line continuation Co-authored-by: ohemorange <ebportnoy@gmail.com>
2020-02-14 20:19:19 -05:00
certbot-nginx
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
install_packages =
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
python {toxinidir}/tools/pip_install_editable.py {[base]all_packages}
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
source_paths =
acme/acme
Refactor certbot/ and certbot/tests/ to use the same structure as the other packages (#7544) Summary of changes in this PR: - Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below). - Make repo root README symlink to `certbot` README. - Pull tests outside of the distributed module. - Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery. - Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments. - Replace mentions of `.` with `certbot` when referring to packages to install, usually editably. - Clean up some unused code around executing tests in a different directory. - Create public shim around main and make that the entry point. New directory structure summary: ``` repo root ("certbot", probably, but for clarity all files I mention are relative to here) ├── certbot │   ├── setup.py │   ├── certbot │   │   ├── __init__.py │   │   ├── achallenges.py │   │   ├── _internal │   │   │   ├── __init__.py │   │   │   ├── account.py │   │   │   ├── ... │   │   ├── ... │   ├── tests │   │   ├── account_test.py │   │   ├── display │   │   │   ├── __init__.py │   │   │   ├── ... │   │   ├── ... # note no __init__.py at this level │   ├── ... ├── acme │   ├── ... ├── certbot-apache │   ├── ... ├── ... ``` * refactor certbot/ and certbot/tests/ to use the same structure as the other packages * git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g" * git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g" * git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g" * Remove replacement of certbot into . in install_and_test.py * copy license back out to main folder * remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/ * Move README back into main folder, and make the version inside certbot/ a symlink * symlink certbot READMEs the other way around * move testdata into the public api certbot zone * update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests * certbot version has been bumped down a directory level * make certbot tests directory not a package and import sibling as module * Remove unused script cruft * change . to certbot in test_sdists * remove outdated comment referencing a command that doesn't work * Install instructions should reference an existing file * update file paths in Dockerfile * some package named in tox.ini were manually specified, change those to certbot * new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code * remove other instance of pyargs * fix up some references in _release.sh by searching for ' . ' and manual check * another stray . in tox.ini * fix paths in tools/_release.sh * Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper * Create public shim around main and make that the entry point * without pyargs, tests cannot be run from an empty directory * Remove cruft for running certbot directly from main * Have main shim take real arg * add docs/api file for main, and fix up main comment * Update certbot/docs/install.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Fix comments in readthedocs requirements files to refer to current package * Update .[docs] reference in contributing.rst * Move plugins tests to certbot tests directory * add certbot tests to MANIFEST.in so packagers can run python setup.py test * move examples directory inside certbot/ * Move CHANGELOG into certbot, and create a top-level symlink * Remove unused sys and logging from main shim * nginx http01 test no longer relies on certbot plugins common test
2019-11-25 17:28:06 -05:00
certbot/certbot
Added certbot-ci to lint section. Silenced and fixed linting warnings. (#8450)
2020-12-16 14:34:12 -05:00
certbot-ci/certbot_integration_tests
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
certbot-apache/certbot_apache
certbot-compatibility-test/certbot_compatibility_test
certbot-dns-cloudflare/certbot_dns_cloudflare
certbot-dns-cloudxns/certbot_dns_cloudxns
certbot-dns-digitalocean/certbot_dns_digitalocean
certbot-dns-dnsimple/certbot_dns_dnsimple
DNS Made Easy DNS Authenticator (#4603) Implement an Authenticator which can fulfill a dns-01 challenge using the DNS Made Easy API. Applicable only for domains using DNS Made Easy. Testing Done: * `tox -e py27` * `tox -e lint` * Manual testing: (`http://api.sandbox.dnsmadeeasy.com/V2.0` used as the `api_endpoint` for all manual testing) * Used `certbot certonly --dns-dnsmadeeasy -d`, specifying a credentials file as a command line argument. Verified that a certificate was successfully obtained without user interaction. * Negative testing: * Path to non-existent credentials file. * Credentials file with unsafe permissions (644). * Path to credentials file with an invalid API key. * Path to credentials file with a malformed API key. * Path to credentials file with an invalid Secret key. * Path to credentials file with a malformed Secret key. * Domain name not registered to DNS Made Easy account.
2017-06-15 19:41:00 -04:00
certbot-dns-dnsmadeeasy/certbot_dns_dnsmadeeasy
Gehirn Infrastracture Service DNS Authenticator (#5702) Implement an Authenticator which can fulfill a dns-01 challenge using the Gehirn DNS (Gehirn Infrastructure Service) API. Applicable only for domains using Gehirn DNS for DNS. Testing Done: * `tox -e py27` * `tox -e lint` * Manual testing: * Used `certbot certonly --dns-gehirn -d`, specifying a credentials file as a command line argument. Verified that a certificate was successfully obtained without user interaction. * Negative testing: * Path to non-existent credentials file. * Credentials file with unsafe permissions (644). * Domain name not registered to Gehirn DNS account.
2018-07-10 20:36:20 -04:00
certbot-dns-gehirn/certbot_dns_gehirn
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
certbot-dns-google/certbot_dns_google
Linode DNS Authenticator (#5302) * Added DNS based authenticator plugin for Linode * Added linode plugin to docs * Added Dockerfile * Added .gitignore and readthedocs.org.requirements.txt * Updated default_propagation_seconds * Updated according to changes requested * Bump version to 0.26.0 * Advertise our packages work on Python 3.7.
2018-07-10 16:51:03 -04:00
certbot-dns-linode/certbot_dns_linode
LuaDNS DNS Authenticator (#4605) Implement an Authenticator which can fulfill a dns-01 challenge using the LuaDNS API. Applicable only for domains using LuaDNS for DNS. Testing Done: * `tox -e py27` * `tox -e lint` * Manual testing: * Used `certbot certonly --dns-luadns -d`, specifying a credentials file as a command line argument. Verified that a certificate was successfully obtained without user interaction. * Negative testing: * Path to non-existent credentials file. * Credentials file with unsafe permissions (644). * Path to credentials file without an email. * Path to credentials file with an invalid email. * Path to credentials file without a token. * Path to credentials file with an invalid token. * Domain name not registered to LuaDNS account.
2017-06-16 17:43:12 -04:00
certbot-dns-luadns/certbot_dns_luadns
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
certbot-dns-nsone/certbot_dns_nsone
OVH DNS Authenticator (#5423) Implement an Authenticator which can fulfill a dns-01 challenge using the OVH DNS API. Applicable only for domains using OVH DNS. Testing Done: * `tox -e py27` * `tox -e lint` * Manual testing: * Used `certbot certonly --dns-ovh -d`, specifying a credentials file as a command line argument. Verified that a certificate was successfully obtained without user interaction. * Used `certbot certonly --dns-ovh -d`, without specifying a credentials file as a command line argument. Verified that the user was prompted and that a certificate was successfully obtained. * Used `certbot certonly -d`. Verified that the user was prompted for a credentials file after selecting dnsimple interactively and that a certificate was successfully obtained. * Used `certbot renew --force-renewal`. Verified that certificates were renewed without user interaction. * Negative testing: * Path to non-existent credentials file. * Credentials file with unsafe permissions (644). * Path to credentials file with an invalid application key. * Path to credentials file with an invalid application secret. * Path to credentials file with an invalid consumer key. * Path to credentials file with missing properties. * Domain name not registered to OVH account.
2018-07-10 23:52:32 -04:00
certbot-dns-ovh/certbot_dns_ovh
RFC 2136 DNS Authenticator (#4701) Introduce a plugin that automates the process of completing a dns-01 challenge by creating, and subsequently removing, TXT records using RFC 2136 Dynamic Updates (a.k.a. nsupdate). This plugin has been tested with BIND, but may work with other RFC 2136-compatible DNS servers, such as PowerDNS.
2017-06-16 19:53:46 -04:00
certbot-dns-rfc2136/certbot_dns_rfc2136
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
certbot-dns-route53/certbot_dns_route53
Sakura Cloud DNS Authenticator (#5701) Implement an Authenticator which can fulfill a dns-01 challenge using the Sakura Cloud DNS API. Applicable only for domains using Sakura Cloud for DNS. Testing Done: * `tox -e py27` * `tox -e lint` * Manual testing: * Used `certbot certonly --dns-sakuracloud -d`, specifying a credentials file as a command line argument. Verified that a certificate was successfully obtained without user interaction. * Negative testing: * Path to non-existent credentials file. * Credentials file with unsafe permissions (644). * Domain name not registered to Sakura Cloud account.
2018-07-10 17:30:37 -04:00
certbot-dns-sakuracloud/certbot_dns_sakuracloud
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
certbot-nginx/certbot_nginx
tests/lock_test.py
Deduplicate package lists in tox.ini (#4608) Use substitution of values form other sections[1] to deduplicate information in tox.ini, including pip install arguments and package paths. 1 - https://tox.readthedocs.io/en/latest/config.html#substitution-for-values-from-other-sections
2017-05-08 14:54:12 -04:00
Remove py26 oldest tests (#4856) * Remove py26 oldest tests. The only systems where we support Python 2.6 use certbot-auto so the oldest supported versions of our dependencies are never used when using supported installation methods. Let's remove this unnecessary and slow test. * Make tox.ini happy * Remove py26-oldest from Travis
2017-06-21 16:50:48 -04:00
[testenv]
Allow to execute a tox target without pinned dependencies (#6590) This PR passes the CERTBOT_NO_PIN environment variable to the unit tests tox envs. By setting CERTBOT_NO_PIN to 1 before executing a given tox env, certbot dependencies will be installed at their latest version instead of the usual pinned version. I also moved the unpin logic one layer below to allow it to be used potentially more widely, and avoid unnecessary merging constraints operation in this case. As warnings are errors now, latest versions of Python will break now the tests, because collections launch a warning when some classes are imported from collections instead of collections.abc. Certbot code is patched, and warning is ignored for now, because a lot of third party libraries still depend on this behavior. * Allow to execute a tox target without pinned dependencies * Correct lint * Retrigger build. * Remove debug code * Added test against unpinned dependencies from test-everything-unpinned-dependencies branch * Remove duplicated assertion to pass TRAVIS and APPVEYOR in default tox environment.
2019-01-17 16:02:35 -05:00
passenv =
CERTBOT_NO_PIN
Write a replacement for pipstrap (#8401) * Add a new, simplified version of pipstrap. * Use tools/pipstrap.py * Uncomment code * Refactor pip_install.py and provide hashes. * Fix test_sdists.sh. * Make code work on Python 2. * Call strip_hashes.py using Python 3. * Pin the oldest version of httplib2 used in distros * Strip enum34 dependency. * Remove pip pinnings from dev_constraints.txt * Correct pipstrap docstring. * Don't set working_dir twice. * Add comments
2020-11-06 05:17:41 -05:00
commands_pre = python {toxinidir}/tools/pipstrap.py
Deduplicate package lists in tox.ini (#4608) Use substitution of values form other sections[1] to deduplicate information in tox.ini, including pip install arguments and package paths. 1 - https://tox.readthedocs.io/en/latest/config.html#substitution-for-values-from-other-sections
2017-05-08 14:54:12 -04:00
commands =
Consolidate cover envs and default to py3-cover (#7905) * Consolidate cover envs and default to py3-cover * use py38 for code coverage in Travis * Disable coverage on Python < 3.6 line.
2020-04-16 11:59:40 -04:00
!cover: {[base]install_and_test} {[base]all_packages}
!cover: python tests/lock_test.py
cover: {[base]install_packages}
cover: {[base]pip_install} certbot-apache[dev]
cover: python tox.cover.py
Set recreate = true in tox.ini. (#7746) Fixes #7745.
2020-02-05 17:37:39 -05:00
# We always recreate the virtual environment to avoid problems like
# https://github.com/certbot/certbot/issues/7745.
recreate = true
Remove py26 oldest tests (#4856) * Remove py26 oldest tests. The only systems where we support Python 2.6 use certbot-auto so the oldest supported versions of our dependencies are never used when using supported installation methods. Let's remove this unnecessary and slow test. * Make tox.ini happy * Remove py26-oldest from Travis
2017-06-21 16:50:48 -04:00
setenv =
Set --pyargs directly in the files where it is needed. (#6727) It was pointed out to me that you can no longer run tox.cover.py directly to run coverage tests on a subset of the packages in this repo. This happened after we did both of: 1. Factored out --pyargs from the different test files and put it in pytest.ini. 2. Moved the options we added to pytest.ini to tox.ini meaning that --pyargs is not set unless you run the file through tox. I think the fact that we factored out --pyargs from the files that needed it was a mistake. --pytest is needed by tox.cover.py and install_and_test.py in order to work correctly. I think CLI options like this which are needed for the file to function should be left in the file directly. Doing anything else in my opinion unnecessarily couples these scripts to other files making them more brittle and harder to maintain. With that said, I also think CLI options which are not needed (such as --numprocesses) can be left to be optionally added through PYTEST_ADDOPTS. * Add --pyargs to tox.cover.py. * Add --pyargs to install_and_test.py. * Remove --pyargs from tox.ini.
2019-01-31 15:57:49 -05:00
PYTEST_ADDOPTS = {env:PYTEST_ADDOPTS:--numprocesses auto}
Remove py26 oldest tests (#4856) * Remove py26 oldest tests. The only systems where we support Python 2.6 use certbot-auto so the oldest supported versions of our dependencies are never used when using supported installation methods. Let's remove this unnecessary and slow test. * Make tox.ini happy * Remove py26-oldest from Travis
2017-06-21 16:50:48 -04:00
PYTHONHASHSEED = 0
[testenv:py27-oldest]
commands =
{[testenv]commands}
setenv =
{[testenv]setenv}
Pin dependencies in oldest tests (#5316) * Add tools/merge_requirements.py * Revert "Fix oldest tests by pinning Google DNS deps (#5000)" This reverts commit f68fba2be2fc342dd72deaaf048ab79e5a8fc2be. * Add tools/oldest_constraints.txt * Remove oldest constraints from tox.ini * Rename dev constraints file * Update tools/pip_install.sh * Update install_and_test.sh * Fix pip_install.sh * Don't cat when you can cp * Add ng-httpsclient to dev constraints for oldest tests * Bump tested setuptools version * Update dev_constraints comment * Better document oldest dependencies * test against oldest versions we say we require * Update dev constraints * Properly handle empty lines * Update constraints gen in pip_install * Remove duplicated zope.component * Reduce pyasn1-modules dependency * Remove blank line * pin back google-api-python-client * pin back uritemplate * pin josepy for oldest tests * Undo changes to install_and_test.sh * Update install_and_test.sh description * use split instead of partition
2017-12-18 15:31:36 -05:00
CERTBOT_OLDEST=1
CloudXNS DNS Authenticator (#4585) Implement an Authenticator which can fulfill a dns-01 challenge using the CloudXNS DNS API. Applicable only for domains using CloudXNS DNS. Testing Done: * `tox -e py27` * `tox -e lint` * Manual testing: * Used `certbot certonly --dns-cloudxns -d`, specifying a credentials file as a command line argument. Verified that a certificate was successfully obtained without user interaction. * Used `certbot certonly --dns-cloudxns -d`, without specifying a credentials file as a command line argument. Verified that the user was prompted and that a certificate was successfully obtained. * Used `certbot certonly -d`. Verified that the user was prompted for a credentials file after selecting cloudxns interactively and that a certificate was successfully obtained. * Used `certbot renew --force-renewal`. Verified that certificates were renewed without user interaction. * Negative testing: * Path to non-existent credentials file. * Credentials file with unsafe permissions (644). * Domain name not registered to CloudXNS account.
2017-05-19 19:23:53 -04:00
Break lockstep between our packages (#5655) Fixes #5490. There's a lot of possibilities discussed in #5490, but I'll try and explain what I actually did here as succinctly as I can. Unfortunately, there's a fair bit to explain. My goal was to break lockstep and give us tests to ensure the minimum specified versions are correct without taking the time now to refactor our whole test setup. To handle specifying each package's minimum acme/certbot version, I added a requirements file to each package. This won't actually be included in the shipped package (because it's not in the MANIFEST). After creating these files and modifying tools/pip_install.sh to use them, I created a separate tox env for most packages (I kept the DNS plugins together for convenience). The reason this is necessary is because we currently use a single environment for each plugin, but if we used this approach for these tests we'd hit issues due to different installed plugins requiring different versions of acme/certbot. There's a lot more discussion about this in #5490 if you're interested in this piece. I unfortunately wasted a lot of time trying to remove the boilerplate this approach causes in tox.ini, but to do this I think we need negations described at complex factor conditions which hasn't made it into a tox release yet. The biggest missing piece here is how to make sure the oldest versions that are currently pinned to master get updated. Currently, they'll stay pinned that way without manual intervention and won't be properly testing the oldest version. I think we should solve this during the larger test/repo refactoring after the release because the tests are using the correct values now and I don't see a simple way around the problem. Once this lands, I'm planning on updating the test-everything tests to do integration tests with the "oldest" versions here. * break lockstep between packages * Use per package requirements files * add local oldest requirements files * update tox.ini * work with dev0 versions * Install requirements in separate step. * don't error when we don't have requirements * install latest packages in editable mode * Update .travis.yml * Add reminder comments * move dev to requirements * request acme[dev] * Update pip_install documentation
2018-03-05 12:50:19 -05:00
[testenv:py27-acme-oldest]
commands =
{[base]install_and_test} acme[dev]
setenv =
{[testenv:py27-oldest]setenv}
[testenv:py27-apache-oldest]
commands =
{[base]install_and_test} certbot-apache
setenv =
{[testenv:py27-oldest]setenv}
[Apache v2] Add apacheconfig as a dependency (#7643) * Add apacheconfig as a dependency. * Change apacheconfig to a dev dependency * Bump apacheconfig dep to 0.3.1
2020-01-07 12:57:43 -05:00
[testenv:py27-apache-v2-oldest]
commands =
{[base]install_and_test} certbot-apache[dev]
setenv =
{[testenv:py27-oldest]setenv}
Break lockstep between our packages (#5655) Fixes #5490. There's a lot of possibilities discussed in #5490, but I'll try and explain what I actually did here as succinctly as I can. Unfortunately, there's a fair bit to explain. My goal was to break lockstep and give us tests to ensure the minimum specified versions are correct without taking the time now to refactor our whole test setup. To handle specifying each package's minimum acme/certbot version, I added a requirements file to each package. This won't actually be included in the shipped package (because it's not in the MANIFEST). After creating these files and modifying tools/pip_install.sh to use them, I created a separate tox env for most packages (I kept the DNS plugins together for convenience). The reason this is necessary is because we currently use a single environment for each plugin, but if we used this approach for these tests we'd hit issues due to different installed plugins requiring different versions of acme/certbot. There's a lot more discussion about this in #5490 if you're interested in this piece. I unfortunately wasted a lot of time trying to remove the boilerplate this approach causes in tox.ini, but to do this I think we need negations described at complex factor conditions which hasn't made it into a tox release yet. The biggest missing piece here is how to make sure the oldest versions that are currently pinned to master get updated. Currently, they'll stay pinned that way without manual intervention and won't be properly testing the oldest version. I think we should solve this during the larger test/repo refactoring after the release because the tests are using the correct values now and I don't see a simple way around the problem. Once this lands, I'm planning on updating the test-everything tests to do integration tests with the "oldest" versions here. * break lockstep between packages * Use per package requirements files * add local oldest requirements files * update tox.ini * work with dev0 versions * Install requirements in separate step. * don't error when we don't have requirements * install latest packages in editable mode * Update .travis.yml * Add reminder comments * move dev to requirements * request acme[dev] * Update pip_install documentation
2018-03-05 12:50:19 -05:00
[testenv:py27-certbot-oldest]
commands =
Refactor certbot/ and certbot/tests/ to use the same structure as the other packages (#7544) Summary of changes in this PR: - Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below). - Make repo root README symlink to `certbot` README. - Pull tests outside of the distributed module. - Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery. - Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments. - Replace mentions of `.` with `certbot` when referring to packages to install, usually editably. - Clean up some unused code around executing tests in a different directory. - Create public shim around main and make that the entry point. New directory structure summary: ``` repo root ("certbot", probably, but for clarity all files I mention are relative to here) ├── certbot │   ├── setup.py │   ├── certbot │   │   ├── __init__.py │   │   ├── achallenges.py │   │   ├── _internal │   │   │   ├── __init__.py │   │   │   ├── account.py │   │   │   ├── ... │   │   ├── ... │   ├── tests │   │   ├── account_test.py │   │   ├── display │   │   │   ├── __init__.py │   │   │   ├── ... │   │   ├── ... # note no __init__.py at this level │   ├── ... ├── acme │   ├── ... ├── certbot-apache │   ├── ... ├── ... ``` * refactor certbot/ and certbot/tests/ to use the same structure as the other packages * git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g" * git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g" * git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g" * Remove replacement of certbot into . in install_and_test.py * copy license back out to main folder * remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/ * Move README back into main folder, and make the version inside certbot/ a symlink * symlink certbot READMEs the other way around * move testdata into the public api certbot zone * update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests * certbot version has been bumped down a directory level * make certbot tests directory not a package and import sibling as module * Remove unused script cruft * change . to certbot in test_sdists * remove outdated comment referencing a command that doesn't work * Install instructions should reference an existing file * update file paths in Dockerfile * some package named in tox.ini were manually specified, change those to certbot * new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code * remove other instance of pyargs * fix up some references in _release.sh by searching for ' . ' and manual check * another stray . in tox.ini * fix paths in tools/_release.sh * Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper * Create public shim around main and make that the entry point * without pyargs, tests cannot be run from an empty directory * Remove cruft for running certbot directly from main * Have main shim take real arg * add docs/api file for main, and fix up main comment * Update certbot/docs/install.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Fix comments in readthedocs requirements files to refer to current package * Update .[docs] reference in contributing.rst * Move plugins tests to certbot tests directory * add certbot tests to MANIFEST.in so packagers can run python setup.py test * move examples directory inside certbot/ * Move CHANGELOG into certbot, and create a top-level symlink * Remove unused sys and logging from main shim * nginx http01 test no longer relies on certbot plugins common test
2019-11-25 17:28:06 -05:00
{[base]install_and_test} certbot[dev]
Break lockstep between our packages (#5655) Fixes #5490. There's a lot of possibilities discussed in #5490, but I'll try and explain what I actually did here as succinctly as I can. Unfortunately, there's a fair bit to explain. My goal was to break lockstep and give us tests to ensure the minimum specified versions are correct without taking the time now to refactor our whole test setup. To handle specifying each package's minimum acme/certbot version, I added a requirements file to each package. This won't actually be included in the shipped package (because it's not in the MANIFEST). After creating these files and modifying tools/pip_install.sh to use them, I created a separate tox env for most packages (I kept the DNS plugins together for convenience). The reason this is necessary is because we currently use a single environment for each plugin, but if we used this approach for these tests we'd hit issues due to different installed plugins requiring different versions of acme/certbot. There's a lot more discussion about this in #5490 if you're interested in this piece. I unfortunately wasted a lot of time trying to remove the boilerplate this approach causes in tox.ini, but to do this I think we need negations described at complex factor conditions which hasn't made it into a tox release yet. The biggest missing piece here is how to make sure the oldest versions that are currently pinned to master get updated. Currently, they'll stay pinned that way without manual intervention and won't be properly testing the oldest version. I think we should solve this during the larger test/repo refactoring after the release because the tests are using the correct values now and I don't see a simple way around the problem. Once this lands, I'm planning on updating the test-everything tests to do integration tests with the "oldest" versions here. * break lockstep between packages * Use per package requirements files * add local oldest requirements files * update tox.ini * work with dev0 versions * Install requirements in separate step. * don't error when we don't have requirements * install latest packages in editable mode * Update .travis.yml * Add reminder comments * move dev to requirements * request acme[dev] * Update pip_install documentation
2018-03-05 12:50:19 -05:00
setenv =
{[testenv:py27-oldest]setenv}
[testenv:py27-dns-oldest]
commands =
{[base]install_and_test} {[base]dns_packages}
setenv =
{[testenv:py27-oldest]setenv}
[testenv:py27-nginx-oldest]
commands =
{[base]install_and_test} certbot-nginx
python tests/lock_test.py
setenv =
{[testenv:py27-oldest]setenv}
test(postfix): env for testing on oldest deps (#6230) Fixes #6124.
2018-07-17 23:00:12 -04:00
Use tox for lint.
2014-11-22 09:12:33 -05:00
[testenv:lint]
Lint certbot code on Python 3, and update Pylint to the latest version (#7551) Part of #7550 This PR makes appropriate corrections to run pylint on Python 3. Why not keeping the dependencies unchanged and just run pylint on Python 3? Because the old version of pylint breaks horribly on Python 3 because of unsupported version of astroid. Why updating pylint + astroid to the latest version ? Because this version only fixes some internal errors occuring during the lint of Certbot code, and is also ready to run gracefully on Python 3.8. Why upgrading mypy ? Because the old version does not support the new version of astroid required to run pylint correctly. Why not upgrading mypy to its latest version ? Because this latest version includes a new typshed version, that adds a lot of new type definitions, and brings dozens of new errors on the Certbot codebase. I would like to fix that in a future PR. That said so, the work has been to find the correct set of new dependency versions, then configure pylint for sane configuration errors in our situation, disable irrelevant lintings errors, then fixing (or ignoring for good reason) the remaining mypy errors. I also made PyLint and MyPy checks run correctly on Windows. * Start configuration * Reconfigure travis * Suspend a check specific to python 3. Start fixing code. * Repair call_args * Fix return + elif lints * Reconfigure development to run mainly on python3 * Remove incompatible Python 3.4 jobs * Suspend pylint in some assertions * Remove pylint in dev * Take first mypy that supports typed-ast>=1.4.0 to limit the migration path * Various return + else lint errors * Find a set of deps that is working with current mypy version * Update local oldest requirements * Remove all current pylint errors * Rebuild letsencrypt-auto * Update mypy to fix pylint with new astroid version, and fix mypy issues * Explain type: ignore * Reconfigure tox, fix none path * Simplify pinning * Remove useless directive * Remove debugging code * Remove continue * Update requirements * Disable unsubscriptable-object check * Disable one check, enabling two more * Plug certbot dev version for oldest requirements * Remove useless disable directives * Remove useless no-member disable * Remove no-else-* checks. Use elif in symetric branches. * Add back assertion * Add new line * Remove unused pylint disable * Remove other pylint disable
2019-12-10 17:12:50 -05:00
basepython = python3
Unified vector loading in letsencrypt.
2015-07-10 09:40:51 -04:00
# separating into multiple invocations disables cross package
# duplicate code checking; if one of the commands fails, others will
# continue, but tox return code will reflect previous error
tox.ini using setup.py dev
2014-11-27 14:13:21 -05:00
commands =
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
{[base]install_packages}
Lint certbot code on Python 3, and update Pylint to the latest version (#7551) Part of #7550 This PR makes appropriate corrections to run pylint on Python 3. Why not keeping the dependencies unchanged and just run pylint on Python 3? Because the old version of pylint breaks horribly on Python 3 because of unsupported version of astroid. Why updating pylint + astroid to the latest version ? Because this version only fixes some internal errors occuring during the lint of Certbot code, and is also ready to run gracefully on Python 3.8. Why upgrading mypy ? Because the old version does not support the new version of astroid required to run pylint correctly. Why not upgrading mypy to its latest version ? Because this latest version includes a new typshed version, that adds a lot of new type definitions, and brings dozens of new errors on the Certbot codebase. I would like to fix that in a future PR. That said so, the work has been to find the correct set of new dependency versions, then configure pylint for sane configuration errors in our situation, disable irrelevant lintings errors, then fixing (or ignoring for good reason) the remaining mypy errors. I also made PyLint and MyPy checks run correctly on Windows. * Start configuration * Reconfigure travis * Suspend a check specific to python 3. Start fixing code. * Repair call_args * Fix return + elif lints * Reconfigure development to run mainly on python3 * Remove incompatible Python 3.4 jobs * Suspend pylint in some assertions * Remove pylint in dev * Take first mypy that supports typed-ast>=1.4.0 to limit the migration path * Various return + else lint errors * Find a set of deps that is working with current mypy version * Update local oldest requirements * Remove all current pylint errors * Rebuild letsencrypt-auto * Update mypy to fix pylint with new astroid version, and fix mypy issues * Explain type: ignore * Reconfigure tox, fix none path * Simplify pinning * Remove useless directive * Remove debugging code * Remove continue * Update requirements * Disable unsubscriptable-object check * Disable one check, enabling two more * Plug certbot dev version for oldest requirements * Remove useless disable directives * Remove useless no-member disable * Remove no-else-* checks. Use elif in symetric branches. * Add back assertion * Add new line * Remove unused pylint disable * Remove other pylint disable
2019-12-10 17:12:50 -05:00
{[base]pip_install} certbot[dev3]
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
python -m pylint --reports=n --rcfile=.pylintrc {[base]source_paths}
Experimentally try travis with the hackish-apache-test
2015-12-16 22:52:35 -05:00
Add mypy (#4386) * Initial configuration of mypy in box, correction of base mypy errors. * Move mypy install to toe * Add pylint comments for typing imports. * Remove typing module for Python 2.6 compatibility.
2017-03-18 22:10:10 -04:00
[testenv:mypy]
Get mypy tox env running in the current setup (#5861) * get mypy tox env running in the current setup * use any python3 with mypy * pin mypy dependencies
2018-04-12 18:47:39 -04:00
basepython = python3
Add mypy (#4386) * Initial configuration of mypy in box, correction of base mypy errors. * Move mypy install to toe * Add pylint comments for typing imports. * Remove typing module for Python 2.6 compatibility.
2017-03-18 22:10:10 -04:00
commands =
Fix oldest tests (#4782) * Add pip_install_editable.sh * add install_and_test.sh * simplify tox.ini and fix oldest tests * Put paths & packages on their own line in tox.ini
2017-06-08 15:22:46 -04:00
{[base]install_packages}
Refactor certbot/ and certbot/tests/ to use the same structure as the other packages (#7544) Summary of changes in this PR: - Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below). - Make repo root README symlink to `certbot` README. - Pull tests outside of the distributed module. - Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery. - Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments. - Replace mentions of `.` with `certbot` when referring to packages to install, usually editably. - Clean up some unused code around executing tests in a different directory. - Create public shim around main and make that the entry point. New directory structure summary: ``` repo root ("certbot", probably, but for clarity all files I mention are relative to here) ├── certbot │   ├── setup.py │   ├── certbot │   │   ├── __init__.py │   │   ├── achallenges.py │   │   ├── _internal │   │   │   ├── __init__.py │   │   │   ├── account.py │   │   │   ├── ... │   │   ├── ... │   ├── tests │   │   ├── account_test.py │   │   ├── display │   │   │   ├── __init__.py │   │   │   ├── ... │   │   ├── ... # note no __init__.py at this level │   ├── ... ├── acme │   ├── ... ├── certbot-apache │   ├── ... ├── ... ``` * refactor certbot/ and certbot/tests/ to use the same structure as the other packages * git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g" * git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g" * git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g" * Remove replacement of certbot into . in install_and_test.py * copy license back out to main folder * remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/ * Move README back into main folder, and make the version inside certbot/ a symlink * symlink certbot READMEs the other way around * move testdata into the public api certbot zone * update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests * certbot version has been bumped down a directory level * make certbot tests directory not a package and import sibling as module * Remove unused script cruft * change . to certbot in test_sdists * remove outdated comment referencing a command that doesn't work * Install instructions should reference an existing file * update file paths in Dockerfile * some package named in tox.ini were manually specified, change those to certbot * new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code * remove other instance of pyargs * fix up some references in _release.sh by searching for ' . ' and manual check * another stray . in tox.ini * fix paths in tools/_release.sh * Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper * Create public shim around main and make that the entry point * without pyargs, tests cannot be run from an empty directory * Remove cruft for running certbot directly from main * Have main shim take real arg * add docs/api file for main, and fix up main comment * Update certbot/docs/install.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Fix comments in readthedocs requirements files to refer to current package * Update .[docs] reference in contributing.rst * Move plugins tests to certbot tests directory * add certbot tests to MANIFEST.in so packagers can run python setup.py test * move examples directory inside certbot/ * Move CHANGELOG into certbot, and create a top-level symlink * Remove unused sys and logging from main shim * nginx http01 test no longer relies on certbot plugins common test
2019-11-25 17:28:06 -05:00
{[base]pip_install} certbot[dev3]
Get mypy running with clean output (#5864) Fixes #5849. * extract mypy flags into mypy.ini file * Get mypy running with clean output
2018-04-12 21:53:07 -04:00
mypy {[base]source_paths}
Add mypy (#4386) * Initial configuration of mypy in box, correction of base mypy errors. * Move mypy install to toe * Add pylint comments for typing imports. * Remove typing module for Python 2.6 compatibility.
2017-03-18 22:10:10 -04:00
Rename hackishapachetest -> apacheconftest Reenable other travis tests as well as this one
2015-12-22 18:50:48 -05:00
[testenv:apacheconftest]
Experimentally try travis with the hackish-apache-test
2015-12-16 22:52:35 -05:00
commands =
Merge snap code into the Certbot repo * merge .gitignore * Move snapcraft.yml up one level. * update source * move test.sh to tox.ini * use new tox.ini in .travis.yml * move snap build code * make script executable * remove unused python3-dev * don't use deprecated classic flag * go back to stable channel * add nginx in snap addons * add deploy steps * Add comments explaining external tox envs. * error if not in CI * don't use --depth * remove old .travis.yml * Add big comment about SNAP_TOKEN. * Set all_branches: true. * Add repo setting. * run travis on tags * Add more documenting comments to .travis.yml.
2020-04-09 16:15:28 -04:00
{[base]pip_install} acme certbot certbot-apache
Refactor tests out of packaged module for apache plugin (#7607) Part of #7593. * Refactor tests out of packaged module for apache plugin * Exclude pycache and .py[cod] * Change tests path in tox.ini
2019-11-27 12:57:35 -05:00
{toxinidir}/certbot-apache/tests/apache-conf-files/apache-conf-test --debian-modules
Stop using staging in apacheconftests (#6647) Fixes #6585. I wrote up three suggestions for fixing this at https://github.com/certbot/certbot/issues/6585#issuecomment-448054502. I took the middle approach of requiring the user to provide an ACME server to use. I like this better than the other approaches which were: > Resolve #5938 instead of this issue. There is value in these tests as is over the compatibility tests in that they don't use Docker and run on different OSes. > Spin up a local Python server to return the directory object. Trying to set up a dummy ACME server seemed hacky and error prone. Other notes about this PR are: * I put the Pebble setup in `tox.ini` rather than `.travis.yml` as this seems much cleaner and more natural. * I created a new `tox` environment called `apacheconftest-with-pebble` that reuses the code from `testenv:apacheconftest` so `apacheconftest` can continue to be used with servers other than Pebble like is done in our test farm tests. * I chose the environment variable `SERVER` for consistency with our integration tests. I chose to not give this environment variable a default but to fail fast when it is not set. * I ran test farm tests on this PR and they passed.
2019-01-09 15:37:45 -05:00
passenv =
SERVER
Merge snap code into the Certbot repo * merge .gitignore * Move snapcraft.yml up one level. * update source * move test.sh to tox.ini * use new tox.ini in .travis.yml * move snap build code * make script executable * remove unused python3-dev * don't use deprecated classic flag * go back to stable channel * add nginx in snap addons * add deploy steps * Add comments explaining external tox envs. * error if not in CI * don't use --depth * remove old .travis.yml * Add big comment about SNAP_TOKEN. * Set all_branches: true. * Add repo setting. * run travis on tags * Add more documenting comments to .travis.yml.
2020-04-09 16:15:28 -04:00
[testenv:apacheconftest-external-with-pebble]
# Run apacheconftest with pebble and Certbot outside of tox's virtual
# environment.
Stop using staging in apacheconftests (#6647) Fixes #6585. I wrote up three suggestions for fixing this at https://github.com/certbot/certbot/issues/6585#issuecomment-448054502. I took the middle approach of requiring the user to provide an ACME server to use. I like this better than the other approaches which were: > Resolve #5938 instead of this issue. There is value in these tests as is over the compatibility tests in that they don't use Docker and run on different OSes. > Spin up a local Python server to return the directory object. Trying to set up a dummy ACME server seemed hacky and error prone. Other notes about this PR are: * I put the Pebble setup in `tox.ini` rather than `.travis.yml` as this seems much cleaner and more natural. * I created a new `tox` environment called `apacheconftest-with-pebble` that reuses the code from `testenv:apacheconftest` so `apacheconftest` can continue to be used with servers other than Pebble like is done in our test farm tests. * I chose the environment variable `SERVER` for consistency with our integration tests. I chose to not give this environment variable a default but to fail fast when it is not set. * I ran test farm tests on this PR and they passed.
2019-01-09 15:37:45 -05:00
commands =
Merge snap code into the Certbot repo * merge .gitignore * Move snapcraft.yml up one level. * update source * move test.sh to tox.ini * use new tox.ini in .travis.yml * move snap build code * make script executable * remove unused python3-dev * don't use deprecated classic flag * go back to stable channel * add nginx in snap addons * add deploy steps * Add comments explaining external tox envs. * error if not in CI * don't use --depth * remove old .travis.yml * Add big comment about SNAP_TOKEN. * Set all_branches: true. * Add repo setting. * run travis on tags * Add more documenting comments to .travis.yml.
2020-04-09 16:15:28 -04:00
{[base]pip_install} certbot-ci
Refactor tests out of packaged module for apache plugin (#7607) Part of #7593. * Refactor tests out of packaged module for apache plugin * Exclude pycache and .py[cod] * Change tests path in tox.ini
2019-11-27 12:57:35 -05:00
{toxinidir}/certbot-apache/tests/apache-conf-files/apache-conf-test-pebble.py --debian-modules
apacheconftest: toxinidir instead of .
2016-04-10 14:44:28 -04:00
Merge snap code into the Certbot repo * merge .gitignore * Move snapcraft.yml up one level. * update source * move test.sh to tox.ini * use new tox.ini in .travis.yml * move snap build code * make script executable * remove unused python3-dev * don't use deprecated classic flag * go back to stable channel * add nginx in snap addons * add deploy steps * Add comments explaining external tox envs. * error if not in CI * don't use --depth * remove old .travis.yml * Add big comment about SNAP_TOKEN. * Set all_branches: true. * Add repo setting. * run travis on tags * Add more documenting comments to .travis.yml.
2020-04-09 16:15:28 -04:00
[testenv:apacheconftest-with-pebble]
commands =
{[base]pip_install} acme certbot certbot-apache
{[testenv:apacheconftest-external-with-pebble]commands}
Add nginxroundtrip to tox
2016-08-10 19:43:15 -04:00
[testenv:nginxroundtrip]
commands =
Refactor certbot/ and certbot/tests/ to use the same structure as the other packages (#7544) Summary of changes in this PR: - Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below). - Make repo root README symlink to `certbot` README. - Pull tests outside of the distributed module. - Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery. - Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments. - Replace mentions of `.` with `certbot` when referring to packages to install, usually editably. - Clean up some unused code around executing tests in a different directory. - Create public shim around main and make that the entry point. New directory structure summary: ``` repo root ("certbot", probably, but for clarity all files I mention are relative to here) ├── certbot │   ├── setup.py │   ├── certbot │   │   ├── __init__.py │   │   ├── achallenges.py │   │   ├── _internal │   │   │   ├── __init__.py │   │   │   ├── account.py │   │   │   ├── ... │   │   ├── ... │   ├── tests │   │   ├── account_test.py │   │   ├── display │   │   │   ├── __init__.py │   │   │   ├── ... │   │   ├── ... # note no __init__.py at this level │   ├── ... ├── acme │   ├── ... ├── certbot-apache │   ├── ... ├── ... ``` * refactor certbot/ and certbot/tests/ to use the same structure as the other packages * git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g" * git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g" * git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g" * Remove replacement of certbot into . in install_and_test.py * copy license back out to main folder * remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/ * Move README back into main folder, and make the version inside certbot/ a symlink * symlink certbot READMEs the other way around * move testdata into the public api certbot zone * update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests * certbot version has been bumped down a directory level * make certbot tests directory not a package and import sibling as module * Remove unused script cruft * change . to certbot in test_sdists * remove outdated comment referencing a command that doesn't work * Install instructions should reference an existing file * update file paths in Dockerfile * some package named in tox.ini were manually specified, change those to certbot * new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code * remove other instance of pyargs * fix up some references in _release.sh by searching for ' . ' and manual check * another stray . in tox.ini * fix paths in tools/_release.sh * Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper * Create public shim around main and make that the entry point * without pyargs, tests cannot be run from an empty directory * Remove cruft for running certbot directly from main * Have main shim take real arg * add docs/api file for main, and fix up main comment * Update certbot/docs/install.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Fix comments in readthedocs requirements files to refer to current package * Update .[docs] reference in contributing.rst * Move plugins tests to certbot tests directory * add certbot tests to MANIFEST.in so packagers can run python setup.py test * move examples directory inside certbot/ * Move CHANGELOG into certbot, and create a top-level symlink * Remove unused sys and logging from main shim * nginx http01 test no longer relies on certbot plugins common test
2019-11-25 17:28:06 -05:00
{[base]pip_install} acme certbot certbot-apache certbot-nginx
Add nginxroundtrip to tox
2016-08-10 19:43:15 -04:00
python certbot-compatibility-test/nginx/roundtrip.py certbot-compatibility-test/nginx/nginx-roundtrip-testdata
Get le-auto tests running on Travis.
2016-01-08 16:55:52 -05:00
Allowing modification check to run using "tox" (#3704) #3337 and #3338 should ideally run when the user type tox. This allows them to catch the problem locally before they make a PR.
2016-10-28 18:05:25 -04:00
# This is a duplication of the command line in testenv:le_auto to
# allow users to run the modification check by running `tox`
[testenv:modification]
commands =
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
python {toxinidir}/tests/modification-check.py
Migrate the CI pipeline from Travis to Azure Pipeline (#8098) Fixes #8071 and fixes https://github.com/certbot/certbot/issues/8110. This PR migrates every job from Travis in Azure Pipeline. This PR essentially converts the Travis jobs into Azure Pipeline with a complete iso-fonctionality (or I made a mistake). The jobs are added in the relevant existing pipelines (`main`, `nightly`, `advanced-test`, `release`). A global refactoring thanks to the templating system is done to reduce greatly the verbosity of the pipeline descriptions. A specific feature (not present in Travis) is added: the stage `On_Failure`. Using directly the Mattermost API, it allows to notify pipeline failure in a Mattermost channel with a link to the failed pipelines without the need to authenticate to Microsoft. See https://github.com/certbot/certbot/pull/8098#issuecomment-649873641 for the post merge actions to do at the end of this work.
2020-07-02 18:01:21 -04:00
passenv = TARGET_BRANCH
Allowing modification check to run using "tox" (#3704) #3337 and #3338 should ideally run when the user type tox. This allows them to catch the problem locally before they make a PR.
2016-10-28 18:05:25 -04:00
add compatibility tests to travis
2016-08-10 19:33:56 -04:00
[testenv:apache_compat]
commands =
docker build -t certbot-compatibility-test -f certbot-compatibility-test/Dockerfile .
docker build -t apache-compat -f certbot-compatibility-test/Dockerfile-apache .
Migrate the CI pipeline from Travis to Azure Pipeline (#8098) Fixes #8071 and fixes https://github.com/certbot/certbot/issues/8110. This PR migrates every job from Travis in Azure Pipeline. This PR essentially converts the Travis jobs into Azure Pipeline with a complete iso-fonctionality (or I made a mistake). The jobs are added in the relevant existing pipelines (`main`, `nightly`, `advanced-test`, `release`). A global refactoring thanks to the templating system is done to reduce greatly the verbosity of the pipeline descriptions. A specific feature (not present in Travis) is added: the stage `On_Failure`. Using directly the Mattermost API, it allows to notify pipeline failure in a Mattermost channel with a link to the failed pipelines without the need to authenticate to Microsoft. See https://github.com/certbot/certbot/pull/8098#issuecomment-649873641 for the post merge actions to do at the end of this work.
2020-07-02 18:01:21 -04:00
docker run --rm -t apache-compat -c apache.tar.gz -vvvv
add compatibility tests to travis
2016-08-10 19:33:56 -04:00
whitelist_externals =
docker
bump pytest-xdist to 1.22.5 (#6253) Signed-off-by: Yoan Blanc <yoan.blanc@exoscale.ch>
2018-07-31 13:31:36 -04:00
passenv =
DOCKER_*
add compatibility tests to travis
2016-08-10 19:33:56 -04:00
[testenv:nginx_compat]
commands =
docker build -t certbot-compatibility-test -f certbot-compatibility-test/Dockerfile .
docker build -t nginx-compat -f certbot-compatibility-test/Dockerfile-nginx .
Migrate the CI pipeline from Travis to Azure Pipeline (#8098) Fixes #8071 and fixes https://github.com/certbot/certbot/issues/8110. This PR migrates every job from Travis in Azure Pipeline. This PR essentially converts the Travis jobs into Azure Pipeline with a complete iso-fonctionality (or I made a mistake). The jobs are added in the relevant existing pipelines (`main`, `nightly`, `advanced-test`, `release`). A global refactoring thanks to the templating system is done to reduce greatly the verbosity of the pipeline descriptions. A specific feature (not present in Travis) is added: the stage `On_Failure`. Using directly the Mattermost API, it allows to notify pipeline failure in a Mattermost channel with a link to the failed pipelines without the need to authenticate to Microsoft. See https://github.com/certbot/certbot/pull/8098#issuecomment-649873641 for the post merge actions to do at the end of this work.
2020-07-02 18:01:21 -04:00
docker run --rm -t nginx-compat -c nginx.tar.gz -vv -aie
add compatibility tests to travis
2016-08-10 19:33:56 -04:00
whitelist_externals =
docker
bump pytest-xdist to 1.22.5 (#6253) Signed-off-by: Yoan Blanc <yoan.blanc@exoscale.ch>
2018-07-31 13:31:36 -04:00
passenv =
DOCKER_*
Expanding tests for le-auto, adding Ubuntu test suite (#2548) * Adding Ubuntu Wily to LEA testing * Setting up certs correctly for Ubuntu 15.10 * Adding 12.04 * Removing redundant update-ca-certificates from 12.04 le-auto testing script * Fixing OpenSSL on Precise * Adding Vivid to le_auto tests * Cleaning up LEA tests configuration for Trusty * Ordering LEA test entries in .travis.yml and renaming them correctly * Removing Ubuntu Vivid * Refining comments * Removing Ubuntu Wily since it reached EOL * Removing .travis.yml duplicates * Fixing nits
2016-11-10 16:15:17 -05:00
add build of Dockerfile-dev (#4717)
2017-05-25 19:27:31 -04:00
[testenv:docker_dev]
# tests the Dockerfile-dev file to ensure development with it works
# as expected
commands =
docker-compose run --rm --service-ports development bash -c 'tox -e lint'
whitelist_externals =
whitelist docker-compose (#6516)
2018-11-30 17:44:47 -05:00
docker-compose
add build of Dockerfile-dev (#4717)
2017-05-25 19:27:31 -04:00
passenv = DOCKER_*
[Unix] Create a framework for certbot integration tests: PART 1 (#6578) * First part * Several optimizations about the docker env setup * Documentation * Various corrections and documentation. Add acme and certbot explicitly as dependencies of certbot-ci. * Correct a variable misinterpreted as a pytest hook * Correct strict parsing option on pebble * Refactor acme setup to be executed from pytest hooks. * Pass TRAVIS env variable to trigger specific xdist logic * Retrigger build. * Work in progress * Config operational * Propagate to xdist * Corrections on acme and misc * Correct subnet for pebble * Remove gobetween, as tls-sni challenges are not tested anymore. * Improve pebble setup. Reduce LOC. * Update acme.py * Optimize acme ca setup, with less temporary assets * Silent setup * Clean code * Remove unused workspace * Use default network driver * Remove bridge * Update package documentation * Remove rerun capability for integration tests, not needed. * Add documentation * Variable for all ports and subnets used by the stack * Update certbot-ci/certbot_integration_tests/conftest.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/acme.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/misc.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update tox.ini Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/misc.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/acme.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/acme.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/conftest.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Rename to acme_server * Add comment * Refactor in a unique context fixture * Remove the need of CERTBOT_ACME_XDIST environment variable * Remove nonstrict/strict options in pebble * Clean dependencies * Clean tox * Change function name * Add comment about coveragerc specificities * Change a comment. * Update setup.py * Update conftest.py * Use the production-ready docker-compose.yml file for Pebble * New style class * Tune pebble to have a stable test environment * Pin a dependency
2019-03-01 16:18:06 -05:00
[testenv:integration]
commands =
Refactor certbot/ and certbot/tests/ to use the same structure as the other packages (#7544) Summary of changes in this PR: - Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below). - Make repo root README symlink to `certbot` README. - Pull tests outside of the distributed module. - Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery. - Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments. - Replace mentions of `.` with `certbot` when referring to packages to install, usually editably. - Clean up some unused code around executing tests in a different directory. - Create public shim around main and make that the entry point. New directory structure summary: ``` repo root ("certbot", probably, but for clarity all files I mention are relative to here) ├── certbot │   ├── setup.py │   ├── certbot │   │   ├── __init__.py │   │   ├── achallenges.py │   │   ├── _internal │   │   │   ├── __init__.py │   │   │   ├── account.py │   │   │   ├── ... │   │   ├── ... │   ├── tests │   │   ├── account_test.py │   │   ├── display │   │   │   ├── __init__.py │   │   │   ├── ... │   │   ├── ... # note no __init__.py at this level │   ├── ... ├── acme │   ├── ... ├── certbot-apache │   ├── ... ├── ... ``` * refactor certbot/ and certbot/tests/ to use the same structure as the other packages * git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g" * git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g" * git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g" * Remove replacement of certbot into . in install_and_test.py * copy license back out to main folder * remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/ * Move README back into main folder, and make the version inside certbot/ a symlink * symlink certbot READMEs the other way around * move testdata into the public api certbot zone * update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests * certbot version has been bumped down a directory level * make certbot tests directory not a package and import sibling as module * Remove unused script cruft * change . to certbot in test_sdists * remove outdated comment referencing a command that doesn't work * Install instructions should reference an existing file * update file paths in Dockerfile * some package named in tox.ini were manually specified, change those to certbot * new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code * remove other instance of pyargs * fix up some references in _release.sh by searching for ' . ' and manual check * another stray . in tox.ini * fix paths in tools/_release.sh * Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper * Create public shim around main and make that the entry point * without pyargs, tests cannot be run from an empty directory * Remove cruft for running certbot directly from main * Have main shim take real arg * add docs/api file for main, and fix up main comment * Update certbot/docs/install.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Fix comments in readthedocs requirements files to refer to current package * Update .[docs] reference in contributing.rst * Move plugins tests to certbot tests directory * add certbot tests to MANIFEST.in so packagers can run python setup.py test * move examples directory inside certbot/ * Move CHANGELOG into certbot, and create a top-level symlink * Remove unused sys and logging from main shim * nginx http01 test no longer relies on certbot plugins common test
2019-11-25 17:28:06 -05:00
{[base]pip_install} acme certbot certbot-nginx certbot-ci
[Unix] Create a framework for certbot integration tests: PART 5-FINAL (#6989) * Connect certbot-ci to travis. Remove old bash files. * Configure test-everything * Protect against import error * Remove unused ignore * Better handling of urllib3 * Correct path * Remove a warning * Correct call * Protect atexit register execution * Update docs/contributing.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update docs/contributing.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Add again some bash scripts to avoid breaking to much retro-compatiblity on third party scripts * Move boulder-v1 and boulder-v2 in nightly tests * Separate oldest unit tests and oldest integration tests * Remove try/except * Test integration included in toxenv * Add a wait to avoid a transient issue on OCSP status in oldest tests * Clean travis.yml, split other tests * Remove useless config * Update .travis.yml Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update tox.ini * Update tox.ini * Remove pytest-sugar * Remove empty pytest.ini, tests are working without it
2019-05-14 16:56:32 -04:00
pytest certbot-ci/certbot_integration_tests \
[Unix] Create a framework for certbot integration tests: PART 1 (#6578) * First part * Several optimizations about the docker env setup * Documentation * Various corrections and documentation. Add acme and certbot explicitly as dependencies of certbot-ci. * Correct a variable misinterpreted as a pytest hook * Correct strict parsing option on pebble * Refactor acme setup to be executed from pytest hooks. * Pass TRAVIS env variable to trigger specific xdist logic * Retrigger build. * Work in progress * Config operational * Propagate to xdist * Corrections on acme and misc * Correct subnet for pebble * Remove gobetween, as tls-sni challenges are not tested anymore. * Improve pebble setup. Reduce LOC. * Update acme.py * Optimize acme ca setup, with less temporary assets * Silent setup * Clean code * Remove unused workspace * Use default network driver * Remove bridge * Update package documentation * Remove rerun capability for integration tests, not needed. * Add documentation * Variable for all ports and subnets used by the stack * Update certbot-ci/certbot_integration_tests/conftest.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/acme.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/misc.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update tox.ini Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/misc.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/acme.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/acme.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/conftest.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Rename to acme_server * Add comment * Refactor in a unique context fixture * Remove the need of CERTBOT_ACME_XDIST environment variable * Remove nonstrict/strict options in pebble * Clean dependencies * Clean tox * Change function name * Add comment about coveragerc specificities * Change a comment. * Update setup.py * Update conftest.py * Use the production-ready docker-compose.yml file for Pebble * New style class * Tune pebble to have a stable test environment * Pin a dependency
2019-03-01 16:18:06 -05:00
--acme-server={env:ACME_SERVER:pebble} \
--cov=acme --cov=certbot --cov=certbot_nginx --cov-report= \
[Unix] Create a framework for certbot integration tests: PART 5-FINAL (#6989) * Connect certbot-ci to travis. Remove old bash files. * Configure test-everything * Protect against import error * Remove unused ignore * Better handling of urllib3 * Correct path * Remove a warning * Correct call * Protect atexit register execution * Update docs/contributing.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update docs/contributing.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Add again some bash scripts to avoid breaking to much retro-compatiblity on third party scripts * Move boulder-v1 and boulder-v2 in nightly tests * Separate oldest unit tests and oldest integration tests * Remove try/except * Test integration included in toxenv * Add a wait to avoid a transient issue on OCSP status in oldest tests * Clean travis.yml, split other tests * Remove useless config * Update .travis.yml Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update tox.ini * Update tox.ini * Remove pytest-sugar * Remove empty pytest.ini, tests are working without it
2019-05-14 16:56:32 -04:00
--cov-config=certbot-ci/certbot_integration_tests/.coveragerc
[Windows|Linux] Forbid os.stat and os.fstat (#7325) Fixes #7212 This PR forbid os.stat and os.fstat, and fix or provide alternatives to avoid its usage in certbot outside of certbot.compat.filesystem. * Reimplement private key mode propagation * Remove other os.stat * Remove last call of os.stat in certbot package * Forbid stat and fstat * Implement mode comparison checks * Add unit tests * Update certbot/compat/filesystem.py Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update certbot/compat/filesystem.py Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Handle case where multiple ace concerns a given SID in has_min_permissions * Add a new test scenario * Add a simple test for has_same_ownership * Fix name function * Add a comment explaining an ACE structure * Move a test in its dedicated class * Improve a message error * Calculate has_min_permission result using effective permission rights to be more generic. * Change an exception message * Add comments, avoid to skip a test. * Update certbot/compat/filesystem.py Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
2019-09-06 17:30:25 -04:00
coverage report --include 'certbot/*' --show-missing --fail-under=65
[Unix] Create a framework for certbot integration tests: PART 4 (#6958) This PR is the part 4 to implement #6541. It adds the integration tests for the nginx certbot plugin, and corresponds to the certbot-ci translation of certbot-nginx/tests/boulder-integration.sh that is executed for each PR. As with certbot core tests, tests are written in Python, and executed by pytest, against a dynamic Boulder/Pebble instance setup. Tests are parallelized, of course, and a specific IntegrationTestsContext class, extended the one from certbot core tests, is crafter for these specific tests: its main goal is to setup a specific nginx instance for the current test. On top of that, I use the test parametrization feature of Pytest, to drastically reduce the size of the actual code: indeed, the 6 tests from the original bash script share the same logic. So using a parametrization, one unique test is written, that is then executed 6 times against 6 different sets of parameters. Note that the module integration_tests.nginx_tests.nginx_config do the same, but in Python, than certbot-nginx/tests/boulder-integration.conf.sh. The latter will be removed in a future PR, with all other bash scripts. * Add nginx tests * Distribute the other_port * Load a pre-generated key/cert for nginx config * Correct preload, remove a test, simplify a variable * Integrate assertion directly in the test function * Check process is not terminated * Add spaces in the nginx config * Add comments * Use indirection * Allow external cert * Add coverage threshold for certbot-nginx
2019-04-23 16:29:48 -04:00
coverage report --include 'certbot-nginx/*' --show-missing --fail-under=74
[Unix] Create a framework for certbot integration tests: PART 5-FINAL (#6989) * Connect certbot-ci to travis. Remove old bash files. * Configure test-everything * Protect against import error * Remove unused ignore * Better handling of urllib3 * Correct path * Remove a warning * Correct call * Protect atexit register execution * Update docs/contributing.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update docs/contributing.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Add again some bash scripts to avoid breaking to much retro-compatiblity on third party scripts * Move boulder-v1 and boulder-v2 in nightly tests * Separate oldest unit tests and oldest integration tests * Remove try/except * Test integration included in toxenv * Add a wait to avoid a transient issue on OCSP status in oldest tests * Clean travis.yml, split other tests * Remove useless config * Update .travis.yml Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update tox.ini * Update tox.ini * Remove pytest-sugar * Remove empty pytest.ini, tests are working without it
2019-05-14 16:56:32 -04:00
passenv = DOCKER_*
Fix integration tests on Windows (#7271) * Fix account_tests * Fix hook executable test * Remove the temporary decorator @broken_on_windows * Fix util_test * No broken unit test on Windows anymore * More elegant mock * Fix context manager * Fix lint * Fix mypy * Adapt coverage * Corrections * Fix lint * Adapt coverage * Update certbot/tests/compat/filesystem_test.py Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update util_test.py * Fix pylint * Forbid os.access * Update os_test.py * Update os.py * Fix lint * Update filesystem.py * Update filesystem.py * Update filesystem.py * Update os.py * Start fixing tests * Platform independent hooks * Fix probe fd close * Add broken_on_windows for integration tests * Fix a lot of tests * Use a python hook script, to prepare cross-platform * New approach to be compliant with Linux and Windows on hook scripts * New tests fixed * Test for permissions on Windows * Permissions comparison for Windows * No broken tests in certbot core anymore * Change mode * Specific config for appveyor * Use forked pebble for now * Various fixes * Assert file permissions for world on private keys * Clean code * Fix several things * Add integration target * Optimize integration env * Re-enable all AppVeyor envs * Use again official pebble * Update pebble_artifacts.py * Set PYTEST_ADDOPTS silently * Update appveyor.yml * Pin pywin32 for tests, give a minimal requirement for certbot. * Remove injection of nginx in PATH * Clean debug code * Various cleanup, ensure to remove workspace after tests * Update tox target * Improve assertions. Control the keyword echoed in hooks * Fix for virtualenv on Python 3.7.4 for Windows * Update certbot-ci/certbot_integration_tests/certbot_tests/assertions.py Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Add conditionally pywin in certbot-ci like in certbot
2019-08-06 18:02:16 -04:00
[testenv:integration-certbot]
commands =
Refactor certbot/ and certbot/tests/ to use the same structure as the other packages (#7544) Summary of changes in this PR: - Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below). - Make repo root README symlink to `certbot` README. - Pull tests outside of the distributed module. - Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery. - Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments. - Replace mentions of `.` with `certbot` when referring to packages to install, usually editably. - Clean up some unused code around executing tests in a different directory. - Create public shim around main and make that the entry point. New directory structure summary: ``` repo root ("certbot", probably, but for clarity all files I mention are relative to here) ├── certbot │   ├── setup.py │   ├── certbot │   │   ├── __init__.py │   │   ├── achallenges.py │   │   ├── _internal │   │   │   ├── __init__.py │   │   │   ├── account.py │   │   │   ├── ... │   │   ├── ... │   ├── tests │   │   ├── account_test.py │   │   ├── display │   │   │   ├── __init__.py │   │   │   ├── ... │   │   ├── ... # note no __init__.py at this level │   ├── ... ├── acme │   ├── ... ├── certbot-apache │   ├── ... ├── ... ``` * refactor certbot/ and certbot/tests/ to use the same structure as the other packages * git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g" * git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g" * git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g" * Remove replacement of certbot into . in install_and_test.py * copy license back out to main folder * remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/ * Move README back into main folder, and make the version inside certbot/ a symlink * symlink certbot READMEs the other way around * move testdata into the public api certbot zone * update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests * certbot version has been bumped down a directory level * make certbot tests directory not a package and import sibling as module * Remove unused script cruft * change . to certbot in test_sdists * remove outdated comment referencing a command that doesn't work * Install instructions should reference an existing file * update file paths in Dockerfile * some package named in tox.ini were manually specified, change those to certbot * new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code * remove other instance of pyargs * fix up some references in _release.sh by searching for ' . ' and manual check * another stray . in tox.ini * fix paths in tools/_release.sh * Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper * Create public shim around main and make that the entry point * without pyargs, tests cannot be run from an empty directory * Remove cruft for running certbot directly from main * Have main shim take real arg * add docs/api file for main, and fix up main comment * Update certbot/docs/install.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Fix comments in readthedocs requirements files to refer to current package * Update .[docs] reference in contributing.rst * Move plugins tests to certbot tests directory * add certbot tests to MANIFEST.in so packagers can run python setup.py test * move examples directory inside certbot/ * Move CHANGELOG into certbot, and create a top-level symlink * Remove unused sys and logging from main shim * nginx http01 test no longer relies on certbot plugins common test
2019-11-25 17:28:06 -05:00
{[base]pip_install} acme certbot certbot-ci
Fix integration tests on Windows (#7271) * Fix account_tests * Fix hook executable test * Remove the temporary decorator @broken_on_windows * Fix util_test * No broken unit test on Windows anymore * More elegant mock * Fix context manager * Fix lint * Fix mypy * Adapt coverage * Corrections * Fix lint * Adapt coverage * Update certbot/tests/compat/filesystem_test.py Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update util_test.py * Fix pylint * Forbid os.access * Update os_test.py * Update os.py * Fix lint * Update filesystem.py * Update filesystem.py * Update filesystem.py * Update os.py * Start fixing tests * Platform independent hooks * Fix probe fd close * Add broken_on_windows for integration tests * Fix a lot of tests * Use a python hook script, to prepare cross-platform * New approach to be compliant with Linux and Windows on hook scripts * New tests fixed * Test for permissions on Windows * Permissions comparison for Windows * No broken tests in certbot core anymore * Change mode * Specific config for appveyor * Use forked pebble for now * Various fixes * Assert file permissions for world on private keys * Clean code * Fix several things * Add integration target * Optimize integration env * Re-enable all AppVeyor envs * Use again official pebble * Update pebble_artifacts.py * Set PYTEST_ADDOPTS silently * Update appveyor.yml * Pin pywin32 for tests, give a minimal requirement for certbot. * Remove injection of nginx in PATH * Clean debug code * Various cleanup, ensure to remove workspace after tests * Update tox target * Improve assertions. Control the keyword echoed in hooks * Fix for virtualenv on Python 3.7.4 for Windows * Update certbot-ci/certbot_integration_tests/certbot_tests/assertions.py Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Add conditionally pywin in certbot-ci like in certbot
2019-08-06 18:02:16 -04:00
pytest certbot-ci/certbot_integration_tests/certbot_tests \
--acme-server={env:ACME_SERVER:pebble} \
--cov=acme --cov=certbot --cov-report= \
--cov-config=certbot-ci/certbot_integration_tests/.coveragerc
coverage report --include 'certbot/*' --show-missing --fail-under=62
Add certbot-dns-rfc2136 integration testing (#8448) * tests: add certbot-dns-rfc2136 integration tests * dont use 'with' form of socket.socket fixes py2 crash * address some feedback: - conftest: make DNS server a global resource - conftest: add dns_xdist parameter into node config - conftest: add --dns-server=bind flag - conftest: if configured, point the ACME server to the DNS server - dnsserver: make it sort-of compatible with xdist (future-proofing) - context: parameterize dns-rfc2136 credentials file (future proofing) - context: reduce dns-rfc2136 propagation time to speed up tests - tox: add a integration-dns-rfc2136 target - rfc2136: add a test/zone for subdelegation - rfc2136: skip tests if no DNS server is configured * try add integration-dns-rfc2136 to CI * mock recursive dns via RPZ * update --dns-server args and tox.ini args * address more feedback: - dns_server: rename rfc2136 creds file to .tpl - dns_server: dont vary dns server port, instead we will vary zone names (#8455) - dns_server: log error if bind9 fails to stop cleanly - dns_server: replace assert with raise - context: remove redundant _worker_id - context: remove redundant cleanup override - context: fix seek/flush in credentials context manager - context: rename skip_if_no_server -> ...bind_server - context: add newline EOF * conftest: document _setup_primary_node sideeffects * ci: rfc2136-integration from standard->nightly * fix _stop_bind (function was renamed to stop) * ignore errors from shutil.rmtree during cleanup * dns_server: check for crash while polling * remove --dry-run from rfc2136 test
2020-11-17 03:27:27 -05:00
[testenv:integration-dns-rfc2136]
commands =
{[base]pip_install} acme certbot certbot-dns-rfc2136 certbot-ci
pytest certbot-ci/certbot_integration_tests/rfc2136_tests \
--acme-server=pebble --dns-server=bind \
add coverage testing to dns-rfc2136 integration (#8469) * add coverage testing to dns-rfc2136 integration * add coverage rule for certbot/* as well
2020-12-06 03:23:33 -05:00
--numprocesses=1 \
--cov=acme --cov=certbot --cov=certbot_dns_rfc2136 --cov-report= \
--cov-config=certbot-ci/certbot_integration_tests/.coveragerc
coverage report --include 'certbot/*' --show-missing --fail-under=45
coverage report --include 'certbot-dns-rfc2136/*' --show-missing --fail-under=87
Add certbot-dns-rfc2136 integration testing (#8448) * tests: add certbot-dns-rfc2136 integration tests * dont use 'with' form of socket.socket fixes py2 crash * address some feedback: - conftest: make DNS server a global resource - conftest: add dns_xdist parameter into node config - conftest: add --dns-server=bind flag - conftest: if configured, point the ACME server to the DNS server - dnsserver: make it sort-of compatible with xdist (future-proofing) - context: parameterize dns-rfc2136 credentials file (future proofing) - context: reduce dns-rfc2136 propagation time to speed up tests - tox: add a integration-dns-rfc2136 target - rfc2136: add a test/zone for subdelegation - rfc2136: skip tests if no DNS server is configured * try add integration-dns-rfc2136 to CI * mock recursive dns via RPZ * update --dns-server args and tox.ini args * address more feedback: - dns_server: rename rfc2136 creds file to .tpl - dns_server: dont vary dns server port, instead we will vary zone names (#8455) - dns_server: log error if bind9 fails to stop cleanly - dns_server: replace assert with raise - context: remove redundant _worker_id - context: remove redundant cleanup override - context: fix seek/flush in credentials context manager - context: rename skip_if_no_server -> ...bind_server - context: add newline EOF * conftest: document _setup_primary_node sideeffects * ci: rfc2136-integration from standard->nightly * fix _stop_bind (function was renamed to stop) * ignore errors from shutil.rmtree during cleanup * dns_server: check for crash while polling * remove --dry-run from rfc2136 test
2020-11-17 03:27:27 -05:00
Merge snap code into the Certbot repo * merge .gitignore * Move snapcraft.yml up one level. * update source * move test.sh to tox.ini * use new tox.ini in .travis.yml * move snap build code * make script executable * remove unused python3-dev * don't use deprecated classic flag * go back to stable channel * add nginx in snap addons * add deploy steps * Add comments explaining external tox envs. * error if not in CI * don't use --depth * remove old .travis.yml * Add big comment about SNAP_TOKEN. * Set all_branches: true. * Add repo setting. * run travis on tags * Add more documenting comments to .travis.yml.
2020-04-09 16:15:28 -04:00
[testenv:integration-external]
# Run integration tests with Certbot outside of tox's virtual environment.
commands =
{[base]pip_install} certbot-ci
pytest certbot-ci/certbot_integration_tests \
--acme-server={env:ACME_SERVER:pebble}
passenv = DOCKER_*
[Unix] Create a framework for certbot integration tests: PART 5-FINAL (#6989) * Connect certbot-ci to travis. Remove old bash files. * Configure test-everything * Protect against import error * Remove unused ignore * Better handling of urllib3 * Correct path * Remove a warning * Correct call * Protect atexit register execution * Update docs/contributing.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update docs/contributing.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Add again some bash scripts to avoid breaking to much retro-compatiblity on third party scripts * Move boulder-v1 and boulder-v2 in nightly tests * Separate oldest unit tests and oldest integration tests * Remove try/except * Test integration included in toxenv * Add a wait to avoid a transient issue on OCSP status in oldest tests * Clean travis.yml, split other tests * Remove useless config * Update .travis.yml Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update tox.ini * Update tox.ini * Remove pytest-sugar * Remove empty pytest.ini, tests are working without it
2019-05-14 16:56:32 -04:00
[testenv:integration-certbot-oldest]
commands =
Refactor certbot/ and certbot/tests/ to use the same structure as the other packages (#7544) Summary of changes in this PR: - Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below). - Make repo root README symlink to `certbot` README. - Pull tests outside of the distributed module. - Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery. - Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments. - Replace mentions of `.` with `certbot` when referring to packages to install, usually editably. - Clean up some unused code around executing tests in a different directory. - Create public shim around main and make that the entry point. New directory structure summary: ``` repo root ("certbot", probably, but for clarity all files I mention are relative to here) ├── certbot │   ├── setup.py │   ├── certbot │   │   ├── __init__.py │   │   ├── achallenges.py │   │   ├── _internal │   │   │   ├── __init__.py │   │   │   ├── account.py │   │   │   ├── ... │   │   ├── ... │   ├── tests │   │   ├── account_test.py │   │   ├── display │   │   │   ├── __init__.py │   │   │   ├── ... │   │   ├── ... # note no __init__.py at this level │   ├── ... ├── acme │   ├── ... ├── certbot-apache │   ├── ... ├── ... ``` * refactor certbot/ and certbot/tests/ to use the same structure as the other packages * git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g" * git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g" * git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g" * Remove replacement of certbot into . in install_and_test.py * copy license back out to main folder * remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/ * Move README back into main folder, and make the version inside certbot/ a symlink * symlink certbot READMEs the other way around * move testdata into the public api certbot zone * update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests * certbot version has been bumped down a directory level * make certbot tests directory not a package and import sibling as module * Remove unused script cruft * change . to certbot in test_sdists * remove outdated comment referencing a command that doesn't work * Install instructions should reference an existing file * update file paths in Dockerfile * some package named in tox.ini were manually specified, change those to certbot * new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code * remove other instance of pyargs * fix up some references in _release.sh by searching for ' . ' and manual check * another stray . in tox.ini * fix paths in tools/_release.sh * Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper * Create public shim around main and make that the entry point * without pyargs, tests cannot be run from an empty directory * Remove cruft for running certbot directly from main * Have main shim take real arg * add docs/api file for main, and fix up main comment * Update certbot/docs/install.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Fix comments in readthedocs requirements files to refer to current package * Update .[docs] reference in contributing.rst * Move plugins tests to certbot tests directory * add certbot tests to MANIFEST.in so packagers can run python setup.py test * move examples directory inside certbot/ * Move CHANGELOG into certbot, and create a top-level symlink * Remove unused sys and logging from main shim * nginx http01 test no longer relies on certbot plugins common test
2019-11-25 17:28:06 -05:00
{[base]pip_install} certbot
[Unix] Create a framework for certbot integration tests: PART 5-FINAL (#6989) * Connect certbot-ci to travis. Remove old bash files. * Configure test-everything * Protect against import error * Remove unused ignore * Better handling of urllib3 * Correct path * Remove a warning * Correct call * Protect atexit register execution * Update docs/contributing.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update docs/contributing.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Add again some bash scripts to avoid breaking to much retro-compatiblity on third party scripts * Move boulder-v1 and boulder-v2 in nightly tests * Separate oldest unit tests and oldest integration tests * Remove try/except * Test integration included in toxenv * Add a wait to avoid a transient issue on OCSP status in oldest tests * Clean travis.yml, split other tests * Remove useless config * Update .travis.yml Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Update tox.ini * Update tox.ini * Remove pytest-sugar * Remove empty pytest.ini, tests are working without it
2019-05-14 16:56:32 -04:00
{[base]pip_install} certbot-ci
pytest certbot-ci/certbot_integration_tests/certbot_tests \
--acme-server={env:ACME_SERVER:pebble}
passenv = DOCKER_*
setenv = {[testenv:py27-oldest]setenv}
[testenv:integration-nginx-oldest]
commands =
{[base]pip_install} certbot-nginx
{[base]pip_install} certbot-ci
pytest certbot-ci/certbot_integration_tests/nginx_tests \
--acme-server={env:ACME_SERVER:pebble}
passenv = DOCKER_*
setenv = {[testenv:py27-oldest]setenv}
Enable test farm tests in Travis (#7072) You can see the full test suite running at https://travis-ci.com/certbot/certbot/builds/112291892. A few noteworthy things: --fast is included because without, the tests would sometimes reach Travis' 50 minute timeout even with 1 test script per Travis build. The only script that is run at release time which is not being run here is https://github.com/certbot/certbot/blob/master/tests/letstest/scripts/test_tests.sh because that script runs tests on the packages installed by certbot-auto which won't be updated until midway through a release. We check TRAVIS_PULL_REQUEST and error out if it is not false for simplicity which should be fine because these tests are never run on PRs. The reason it's more complex to run test farm tests on PRs is the test farm tests need a named branch to pull from and Travis effectively merges the PR into the target branch before running tests complicating this. I don't think this should block this PRs, but the one final change we may want to make to the current setup is #7071. * Add encrypted private key. * Add test farm tests to tox and travis. * Change magic profile name. * Further split test farm tests. * Build local branch. * more depth
2019-05-17 19:30:20 -04:00
Migrate the CI pipeline from Travis to Azure Pipeline (#8098) Fixes #8071 and fixes https://github.com/certbot/certbot/issues/8110. This PR migrates every job from Travis in Azure Pipeline. This PR essentially converts the Travis jobs into Azure Pipeline with a complete iso-fonctionality (or I made a mistake). The jobs are added in the relevant existing pipelines (`main`, `nightly`, `advanced-test`, `release`). A global refactoring thanks to the templating system is done to reduce greatly the verbosity of the pipeline descriptions. A specific feature (not present in Travis) is added: the stage `On_Failure`. Using directly the Mattermost API, it allows to notify pipeline failure in a Mattermost channel with a link to the failed pipelines without the need to authenticate to Microsoft. See https://github.com/certbot/certbot/pull/8098#issuecomment-649873641 for the post merge actions to do at the end of this work.
2020-07-02 18:01:21 -04:00
[testenv:test-farm-tests-base]
Enable test farm tests in Travis (#7072) You can see the full test suite running at https://travis-ci.com/certbot/certbot/builds/112291892. A few noteworthy things: --fast is included because without, the tests would sometimes reach Travis' 50 minute timeout even with 1 test script per Travis build. The only script that is run at release time which is not being run here is https://github.com/certbot/certbot/blob/master/tests/letstest/scripts/test_tests.sh because that script runs tests on the packages installed by certbot-auto which won't be updated until midway through a release. We check TRAVIS_PULL_REQUEST and error out if it is not false for simplicity which should be fine because these tests are never run on PRs. The reason it's more complex to run test farm tests on PRs is the test farm tests need a named branch to pull from and Travis effectively merges the PR into the target branch before running tests complicating this. I don't think this should block this PRs, but the one final change we may want to make to the current setup is #7071. * Add encrypted private key. * Add test farm tests to tox and travis. * Change magic profile name. * Further split test farm tests. * Build local branch. * more depth
2019-05-17 19:30:20 -04:00
changedir = tests/letstest
deps = -rtests/letstest/requirements.txt
passenv =
AWS_*
setenv = AWS_DEFAULT_REGION=us-east-1
Migrate the CI pipeline from Travis to Azure Pipeline (#8098) Fixes #8071 and fixes https://github.com/certbot/certbot/issues/8110. This PR migrates every job from Travis in Azure Pipeline. This PR essentially converts the Travis jobs into Azure Pipeline with a complete iso-fonctionality (or I made a mistake). The jobs are added in the relevant existing pipelines (`main`, `nightly`, `advanced-test`, `release`). A global refactoring thanks to the templating system is done to reduce greatly the verbosity of the pipeline descriptions. A specific feature (not present in Travis) is added: the stage `On_Failure`. Using directly the Mattermost API, it allows to notify pipeline failure in a Mattermost channel with a link to the failed pipelines without the need to authenticate to Microsoft. See https://github.com/certbot/certbot/pull/8098#issuecomment-649873641 for the post merge actions to do at the end of this work.
2020-07-02 18:01:21 -04:00
[testenv:test-farm-apache2]
changedir = {[testenv:test-farm-tests-base]changedir}
Automatically retry test farm tests (#8325) Fixes #8317. * move retry to script * Retry test farm tests. * Fix retry path.
2020-09-30 20:05:52 -04:00
commands = {toxinidir}/tools/retry.sh python multitester.py apache2_targets.yaml {env:AWS_EC2_PEM_FILE} SET_BY_ENV scripts/test_apache2.sh --repo {toxinidir}
Migrate the CI pipeline from Travis to Azure Pipeline (#8098) Fixes #8071 and fixes https://github.com/certbot/certbot/issues/8110. This PR migrates every job from Travis in Azure Pipeline. This PR essentially converts the Travis jobs into Azure Pipeline with a complete iso-fonctionality (or I made a mistake). The jobs are added in the relevant existing pipelines (`main`, `nightly`, `advanced-test`, `release`). A global refactoring thanks to the templating system is done to reduce greatly the verbosity of the pipeline descriptions. A specific feature (not present in Travis) is added: the stage `On_Failure`. Using directly the Mattermost API, it allows to notify pipeline failure in a Mattermost channel with a link to the failed pipelines without the need to authenticate to Microsoft. See https://github.com/certbot/certbot/pull/8098#issuecomment-649873641 for the post merge actions to do at the end of this work.
2020-07-02 18:01:21 -04:00
deps = {[testenv:test-farm-tests-base]deps}
passenv = {[testenv:test-farm-tests-base]passenv}
setenv = {[testenv:test-farm-tests-base]setenv}
[testenv:test-farm-leauto-upgrades]
changedir = {[testenv:test-farm-tests-base]changedir}
Automatically retry test farm tests (#8325) Fixes #8317. * move retry to script * Retry test farm tests. * Fix retry path.
2020-09-30 20:05:52 -04:00
commands = {toxinidir}/tools/retry.sh python multitester.py auto_targets.yaml {env:AWS_EC2_PEM_FILE} SET_BY_ENV scripts/test_leauto_upgrades.sh --repo {toxinidir}
Migrate the CI pipeline from Travis to Azure Pipeline (#8098) Fixes #8071 and fixes https://github.com/certbot/certbot/issues/8110. This PR migrates every job from Travis in Azure Pipeline. This PR essentially converts the Travis jobs into Azure Pipeline with a complete iso-fonctionality (or I made a mistake). The jobs are added in the relevant existing pipelines (`main`, `nightly`, `advanced-test`, `release`). A global refactoring thanks to the templating system is done to reduce greatly the verbosity of the pipeline descriptions. A specific feature (not present in Travis) is added: the stage `On_Failure`. Using directly the Mattermost API, it allows to notify pipeline failure in a Mattermost channel with a link to the failed pipelines without the need to authenticate to Microsoft. See https://github.com/certbot/certbot/pull/8098#issuecomment-649873641 for the post merge actions to do at the end of this work.
2020-07-02 18:01:21 -04:00
deps = {[testenv:test-farm-tests-base]deps}
passenv = {[testenv:test-farm-tests-base]passenv}
setenv = {[testenv:test-farm-tests-base]setenv}
[testenv:test-farm-certonly-standalone]
changedir = {[testenv:test-farm-tests-base]changedir}
Automatically retry test farm tests (#8325) Fixes #8317. * move retry to script * Retry test farm tests. * Fix retry path.
2020-09-30 20:05:52 -04:00
commands = {toxinidir}/tools/retry.sh python multitester.py auto_targets.yaml {env:AWS_EC2_PEM_FILE} SET_BY_ENV scripts/test_letsencrypt_auto_certonly_standalone.sh --repo {toxinidir}
Migrate the CI pipeline from Travis to Azure Pipeline (#8098) Fixes #8071 and fixes https://github.com/certbot/certbot/issues/8110. This PR migrates every job from Travis in Azure Pipeline. This PR essentially converts the Travis jobs into Azure Pipeline with a complete iso-fonctionality (or I made a mistake). The jobs are added in the relevant existing pipelines (`main`, `nightly`, `advanced-test`, `release`). A global refactoring thanks to the templating system is done to reduce greatly the verbosity of the pipeline descriptions. A specific feature (not present in Travis) is added: the stage `On_Failure`. Using directly the Mattermost API, it allows to notify pipeline failure in a Mattermost channel with a link to the failed pipelines without the need to authenticate to Microsoft. See https://github.com/certbot/certbot/pull/8098#issuecomment-649873641 for the post merge actions to do at the end of this work.
2020-07-02 18:01:21 -04:00
deps = {[testenv:test-farm-tests-base]deps}
passenv = {[testenv:test-farm-tests-base]passenv}
setenv = {[testenv:test-farm-tests-base]setenv}
[testenv:test-farm-sdists]
changedir = {[testenv:test-farm-tests-base]changedir}
Automatically retry test farm tests (#8325) Fixes #8317. * move retry to script * Retry test farm tests. * Fix retry path.
2020-09-30 20:05:52 -04:00
commands = {toxinidir}/tools/retry.sh python multitester.py targets.yaml {env:AWS_EC2_PEM_FILE} SET_BY_ENV scripts/test_sdists.sh --repo {toxinidir}
Migrate the CI pipeline from Travis to Azure Pipeline (#8098) Fixes #8071 and fixes https://github.com/certbot/certbot/issues/8110. This PR migrates every job from Travis in Azure Pipeline. This PR essentially converts the Travis jobs into Azure Pipeline with a complete iso-fonctionality (or I made a mistake). The jobs are added in the relevant existing pipelines (`main`, `nightly`, `advanced-test`, `release`). A global refactoring thanks to the templating system is done to reduce greatly the verbosity of the pipeline descriptions. A specific feature (not present in Travis) is added: the stage `On_Failure`. Using directly the Mattermost API, it allows to notify pipeline failure in a Mattermost channel with a link to the failed pipelines without the need to authenticate to Microsoft. See https://github.com/certbot/certbot/pull/8098#issuecomment-649873641 for the post merge actions to do at the end of this work.
2020-07-02 18:01:21 -04:00
deps = {[testenv:test-farm-tests-base]deps}
passenv = {[testenv:test-farm-tests-base]passenv}
setenv = {[testenv:test-farm-tests-base]setenv}
Reference in a new issue Copy permalink