certbot/tox.ini

266 lines
7.7 KiB
INI
Raw Permalink Normal View History

2015-01-28 07:56:12 -05:00
# Tox (http://tox.testrun.org/) is a tool for running tests in
# multiple virtualenvs. To use it, "pip install tox" and then run
# "tox" from this directory.
2014-11-22 09:02:52 -05:00
[tox]
skipsdist = true
2018-12-17 19:22:03 -05:00
envlist = modification,py3,py27-cover,lint,mypy
[base]
# pip installs the requested packages in editable mode
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
pip_install = python {toxinidir}/tools/pip_install_editable.py
# pip installs the requested packages in editable mode and runs unit tests on
# them. Each package is installed and tested in the order they are provided
# before the script moves on to the next package. All dependencies are pinned
# to a specific version for increased stability for developers.
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
install_and_test = python {toxinidir}/tools/install_and_test.py
dns_packages =
certbot-dns-cloudflare \
certbot-dns-cloudxns \
certbot-dns-digitalocean \
certbot-dns-dnsimple \
certbot-dns-dnsmadeeasy \
certbot-dns-gehirn \
certbot-dns-google \
certbot-dns-linode \
certbot-dns-luadns \
certbot-dns-nsone \
certbot-dns-ovh \
certbot-dns-rfc2136 \
certbot-dns-route53 \
certbot-dns-sakuracloud
all_packages =
Break lockstep between our packages (#5655) Fixes #5490. There's a lot of possibilities discussed in #5490, but I'll try and explain what I actually did here as succinctly as I can. Unfortunately, there's a fair bit to explain. My goal was to break lockstep and give us tests to ensure the minimum specified versions are correct without taking the time now to refactor our whole test setup. To handle specifying each package's minimum acme/certbot version, I added a requirements file to each package. This won't actually be included in the shipped package (because it's not in the MANIFEST). After creating these files and modifying tools/pip_install.sh to use them, I created a separate tox env for most packages (I kept the DNS plugins together for convenience). The reason this is necessary is because we currently use a single environment for each plugin, but if we used this approach for these tests we'd hit issues due to different installed plugins requiring different versions of acme/certbot. There's a lot more discussion about this in #5490 if you're interested in this piece. I unfortunately wasted a lot of time trying to remove the boilerplate this approach causes in tox.ini, but to do this I think we need negations described at complex factor conditions which hasn't made it into a tox release yet. The biggest missing piece here is how to make sure the oldest versions that are currently pinned to master get updated. Currently, they'll stay pinned that way without manual intervention and won't be properly testing the oldest version. I think we should solve this during the larger test/repo refactoring after the release because the tests are using the correct values now and I don't see a simple way around the problem. Once this lands, I'm planning on updating the test-everything tests to do integration tests with the "oldest" versions here. * break lockstep between packages * Use per package requirements files * add local oldest requirements files * update tox.ini * work with dev0 versions * Install requirements in separate step. * don't error when we don't have requirements * install latest packages in editable mode * Update .travis.yml * Add reminder comments * move dev to requirements * request acme[dev] * Update pip_install documentation
2018-03-05 12:50:19 -05:00
acme[dev] \
.[dev] \
certbot-apache \
{[base]dns_packages} \
certbot-nginx \
certbot-postfix \
letshelp-certbot
install_packages =
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
python {toxinidir}/tools/pip_install_editable.py {[base]all_packages}
source_paths =
acme/acme
certbot
certbot-apache/certbot_apache
certbot-compatibility-test/certbot_compatibility_test
certbot-dns-cloudflare/certbot_dns_cloudflare
certbot-dns-cloudxns/certbot_dns_cloudxns
certbot-dns-digitalocean/certbot_dns_digitalocean
certbot-dns-dnsimple/certbot_dns_dnsimple
certbot-dns-dnsmadeeasy/certbot_dns_dnsmadeeasy
certbot-dns-gehirn/certbot_dns_gehirn
certbot-dns-google/certbot_dns_google
certbot-dns-linode/certbot_dns_linode
certbot-dns-luadns/certbot_dns_luadns
certbot-dns-nsone/certbot_dns_nsone
certbot-dns-ovh/certbot_dns_ovh
certbot-dns-rfc2136/certbot_dns_rfc2136
certbot-dns-route53/certbot_dns_route53
certbot-dns-sakuracloud/certbot_dns_sakuracloud
certbot-nginx/certbot_nginx
certbot-postfix/certbot_postfix
letshelp-certbot/letshelp_certbot
tests/lock_test.py
[testenv]
passenv =
CERTBOT_NO_PIN
commands =
{[base]install_and_test} {[base]all_packages}
python tests/lock_test.py
setenv =
PYTEST_ADDOPTS = {env:PYTEST_ADDOPTS:--numprocesses auto}
PYTHONHASHSEED = 0
[testenv:py27-oldest]
commands =
{[testenv]commands}
setenv =
{[testenv]setenv}
CERTBOT_OLDEST=1
Break lockstep between our packages (#5655) Fixes #5490. There's a lot of possibilities discussed in #5490, but I'll try and explain what I actually did here as succinctly as I can. Unfortunately, there's a fair bit to explain. My goal was to break lockstep and give us tests to ensure the minimum specified versions are correct without taking the time now to refactor our whole test setup. To handle specifying each package's minimum acme/certbot version, I added a requirements file to each package. This won't actually be included in the shipped package (because it's not in the MANIFEST). After creating these files and modifying tools/pip_install.sh to use them, I created a separate tox env for most packages (I kept the DNS plugins together for convenience). The reason this is necessary is because we currently use a single environment for each plugin, but if we used this approach for these tests we'd hit issues due to different installed plugins requiring different versions of acme/certbot. There's a lot more discussion about this in #5490 if you're interested in this piece. I unfortunately wasted a lot of time trying to remove the boilerplate this approach causes in tox.ini, but to do this I think we need negations described at complex factor conditions which hasn't made it into a tox release yet. The biggest missing piece here is how to make sure the oldest versions that are currently pinned to master get updated. Currently, they'll stay pinned that way without manual intervention and won't be properly testing the oldest version. I think we should solve this during the larger test/repo refactoring after the release because the tests are using the correct values now and I don't see a simple way around the problem. Once this lands, I'm planning on updating the test-everything tests to do integration tests with the "oldest" versions here. * break lockstep between packages * Use per package requirements files * add local oldest requirements files * update tox.ini * work with dev0 versions * Install requirements in separate step. * don't error when we don't have requirements * install latest packages in editable mode * Update .travis.yml * Add reminder comments * move dev to requirements * request acme[dev] * Update pip_install documentation
2018-03-05 12:50:19 -05:00
[testenv:py27-acme-oldest]
commands =
{[base]install_and_test} acme[dev]
setenv =
{[testenv:py27-oldest]setenv}
[testenv:py27-apache-oldest]
commands =
{[base]install_and_test} certbot-apache
setenv =
{[testenv:py27-oldest]setenv}
[testenv:py27-certbot-oldest]
commands =
{[base]install_and_test} .[dev]
setenv =
{[testenv:py27-oldest]setenv}
[testenv:py27-dns-oldest]
commands =
{[base]install_and_test} {[base]dns_packages}
setenv =
{[testenv:py27-oldest]setenv}
[testenv:py27-nginx-oldest]
commands =
{[base]install_and_test} certbot-nginx
python tests/lock_test.py
setenv =
{[testenv:py27-oldest]setenv}
[testenv:py27-postfix-oldest]
commands =
{[base]install_and_test} certbot-postfix
setenv =
{[testenv:py27-oldest]setenv}
[testenv:py27_install]
basepython = python2.7
commands =
{[base]install_packages}
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
[testenv:py27-cover]
2015-01-28 07:56:12 -05:00
basepython = python2.7
2014-11-27 14:30:56 -05:00
commands =
{[base]install_packages}
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
python tox.cover.py
[testenv:py37-cover]
basepython = python3.7
commands =
{[base]install_packages}
python tox.cover.py
2014-11-27 14:30:56 -05:00
2014-11-22 09:12:33 -05:00
[testenv:lint]
2015-01-28 07:56:12 -05:00
basepython = python2.7
2015-07-10 09:40:51 -04:00
# separating into multiple invocations disables cross package
# duplicate code checking; if one of the commands fails, others will
# continue, but tox return code will reflect previous error
2014-11-27 14:13:21 -05:00
commands =
{[base]install_packages}
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
python -m pylint --reports=n --rcfile=.pylintrc {[base]source_paths}
[testenv:mypy]
basepython = python3
commands =
{[base]install_packages}
{[base]pip_install} .[dev3]
mypy {[base]source_paths}
[testenv:apacheconftest]
#basepython = python2.7
commands =
{[base]pip_install} acme . certbot-apache certbot-compatibility-test
{toxinidir}/certbot-apache/certbot_apache/tests/apache-conf-files/apache-conf-test --debian-modules
passenv =
SERVER
[testenv:apacheconftest-with-pebble]
commands =
{toxinidir}/tests/pebble-fetch.sh
{[testenv:apacheconftest]commands}
passenv =
HOME
GOPATH
PEBBLEPATH
setenv =
SERVER=https://localhost:14000/dir
2016-04-10 14:44:28 -04:00
2016-08-10 19:43:15 -04:00
[testenv:nginxroundtrip]
commands =
{[base]pip_install} acme . certbot-apache certbot-nginx
2016-08-10 19:43:15 -04:00
python certbot-compatibility-test/nginx/roundtrip.py certbot-compatibility-test/nginx/nginx-roundtrip-testdata
2016-01-08 16:55:52 -05:00
# This is a duplication of the command line in testenv:le_auto to
# allow users to run the modification check by running `tox`
[testenv:modification]
commands =
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
python {toxinidir}/tests/modification-check.py
2016-08-10 19:33:56 -04:00
[testenv:apache_compat]
commands =
docker build -t certbot-compatibility-test -f certbot-compatibility-test/Dockerfile .
docker build -t apache-compat -f certbot-compatibility-test/Dockerfile-apache .
docker run --rm -it apache-compat -c apache.tar.gz -vvvv
whitelist_externals =
docker
passenv =
DOCKER_*
2016-08-10 19:33:56 -04:00
[testenv:nginx_compat]
commands =
docker build -t certbot-compatibility-test -f certbot-compatibility-test/Dockerfile .
docker build -t nginx-compat -f certbot-compatibility-test/Dockerfile-nginx .
docker run --rm -it nginx-compat -c nginx.tar.gz -vv -aie
2016-08-10 19:33:56 -04:00
whitelist_externals =
docker
passenv =
DOCKER_*
[testenv:le_auto_trusty]
# At the moment, this tests under Python 2.7 only, as only that version is
# readily available on the Trusty Docker image.
commands =
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
python {toxinidir}/tests/modification-check.py
docker build -f letsencrypt-auto-source/Dockerfile.trusty -t lea letsencrypt-auto-source
docker run --rm -t -i lea
whitelist_externals =
docker
passenv =
DOCKER_*
TRAVIS_BRANCH
[testenv:le_auto_xenial]
# At the moment, this tests under Python 2.7 only.
commands =
docker build -f letsencrypt-auto-source/Dockerfile.xenial -t lea letsencrypt-auto-source
docker run --rm -t -i lea
whitelist_externals =
docker
passenv = DOCKER_*
[testenv:le_auto_jessie]
# At the moment, this tests under Python 2.7 only, as only that version is
# readily available on the Wheezy Docker image.
commands =
docker build -f letsencrypt-auto-source/Dockerfile.jessie -t lea letsencrypt-auto-source
docker run --rm -t -i lea
whitelist_externals =
docker
passenv = DOCKER_*
[testenv:le_auto_centos6]
# At the moment, this tests under Python 2.6 only, as only that version is
# readily available on the CentOS 6 Docker image.
commands =
docker build -f letsencrypt-auto-source/Dockerfile.centos6 -t lea letsencrypt-auto-source
docker run --rm -t -i lea
whitelist_externals =
docker
passenv = DOCKER_*
2017-05-25 19:27:31 -04:00
[testenv:docker_dev]
# tests the Dockerfile-dev file to ensure development with it works
# as expected
commands =
docker-compose run --rm --service-ports development bash -c 'tox -e lint'
whitelist_externals =
2018-11-30 17:44:47 -05:00
docker-compose
2017-05-25 19:27:31 -04:00
passenv = DOCKER_*
[Unix] Create a framework for certbot integration tests: PART 1 (#6578) * First part * Several optimizations about the docker env setup * Documentation * Various corrections and documentation. Add acme and certbot explicitly as dependencies of certbot-ci. * Correct a variable misinterpreted as a pytest hook * Correct strict parsing option on pebble * Refactor acme setup to be executed from pytest hooks. * Pass TRAVIS env variable to trigger specific xdist logic * Retrigger build. * Work in progress * Config operational * Propagate to xdist * Corrections on acme and misc * Correct subnet for pebble * Remove gobetween, as tls-sni challenges are not tested anymore. * Improve pebble setup. Reduce LOC. * Update acme.py * Optimize acme ca setup, with less temporary assets * Silent setup * Clean code * Remove unused workspace * Use default network driver * Remove bridge * Update package documentation * Remove rerun capability for integration tests, not needed. * Add documentation * Variable for all ports and subnets used by the stack * Update certbot-ci/certbot_integration_tests/conftest.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/acme.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/misc.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update tox.ini Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/misc.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/acme.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/acme.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/conftest.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Rename to acme_server * Add comment * Refactor in a unique context fixture * Remove the need of CERTBOT_ACME_XDIST environment variable * Remove nonstrict/strict options in pebble * Clean dependencies * Clean tox * Change function name * Add comment about coveragerc specificities * Change a comment. * Update setup.py * Update conftest.py * Use the production-ready docker-compose.yml file for Pebble * New style class * Tune pebble to have a stable test environment * Pin a dependency
2019-03-01 16:18:06 -05:00
[testenv:integration]
commands =
{[base]pip_install} acme . certbot-nginx certbot-ci
pytest {toxinidir}/certbot-ci/certbot_integration_tests \
--acme-server={env:ACME_SERVER:pebble} \
--cov=acme --cov=certbot --cov=certbot_nginx --cov-report= \
--cov-config={toxinidir}/certbot-ci/certbot_integration_tests/.coveragerc \
-W 'ignore:Unverified HTTPS request'
coverage report --include 'certbot/*' --show-missing --fail-under=64
[Unix] Create a framework for certbot integration tests: PART 4 (#6958) This PR is the part 4 to implement #6541. It adds the integration tests for the nginx certbot plugin, and corresponds to the certbot-ci translation of certbot-nginx/tests/boulder-integration.sh that is executed for each PR. As with certbot core tests, tests are written in Python, and executed by pytest, against a dynamic Boulder/Pebble instance setup. Tests are parallelized, of course, and a specific IntegrationTestsContext class, extended the one from certbot core tests, is crafter for these specific tests: its main goal is to setup a specific nginx instance for the current test. On top of that, I use the test parametrization feature of Pytest, to drastically reduce the size of the actual code: indeed, the 6 tests from the original bash script share the same logic. So using a parametrization, one unique test is written, that is then executed 6 times against 6 different sets of parameters. Note that the module integration_tests.nginx_tests.nginx_config do the same, but in Python, than certbot-nginx/tests/boulder-integration.conf.sh. The latter will be removed in a future PR, with all other bash scripts. * Add nginx tests * Distribute the other_port * Load a pre-generated key/cert for nginx config * Correct preload, remove a test, simplify a variable * Integrate assertion directly in the test function * Check process is not terminated * Add spaces in the nginx config * Add comments * Use indirection * Allow external cert * Add coverage threshold for certbot-nginx
2019-04-23 16:29:48 -04:00
coverage report --include 'certbot-nginx/*' --show-missing --fail-under=74
[Unix] Create a framework for certbot integration tests: PART 1 (#6578) * First part * Several optimizations about the docker env setup * Documentation * Various corrections and documentation. Add acme and certbot explicitly as dependencies of certbot-ci. * Correct a variable misinterpreted as a pytest hook * Correct strict parsing option on pebble * Refactor acme setup to be executed from pytest hooks. * Pass TRAVIS env variable to trigger specific xdist logic * Retrigger build. * Work in progress * Config operational * Propagate to xdist * Corrections on acme and misc * Correct subnet for pebble * Remove gobetween, as tls-sni challenges are not tested anymore. * Improve pebble setup. Reduce LOC. * Update acme.py * Optimize acme ca setup, with less temporary assets * Silent setup * Clean code * Remove unused workspace * Use default network driver * Remove bridge * Update package documentation * Remove rerun capability for integration tests, not needed. * Add documentation * Variable for all ports and subnets used by the stack * Update certbot-ci/certbot_integration_tests/conftest.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/acme.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/misc.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update tox.ini Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/misc.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/acme.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/utils/acme.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Update certbot-ci/certbot_integration_tests/conftest.py Co-Authored-By: adferrand <adferrand@users.noreply.github.com> * Rename to acme_server * Add comment * Refactor in a unique context fixture * Remove the need of CERTBOT_ACME_XDIST environment variable * Remove nonstrict/strict options in pebble * Clean dependencies * Clean tox * Change function name * Add comment about coveragerc specificities * Change a comment. * Update setup.py * Update conftest.py * Use the production-ready docker-compose.yml file for Pebble * New style class * Tune pebble to have a stable test environment * Pin a dependency
2019-03-01 16:18:06 -05:00
passenv =
DOCKER_*