Commit graph

28 commits

Author SHA1 Message Date
Adrien Ferrand
b67fda8832
Fix integration tests on Windows (#7271)
* Fix account_tests

* Fix hook executable test

* Remove the temporary decorator @broken_on_windows

* Fix util_test

* No broken unit test on Windows anymore

* More elegant mock

* Fix context manager

* Fix lint

* Fix mypy

* Adapt coverage

* Corrections

* Fix lint

* Adapt coverage

* Update certbot/tests/compat/filesystem_test.py

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update util_test.py

* Fix pylint

* Forbid os.access

* Update os_test.py

* Update os.py

* Fix lint

* Update filesystem.py

* Update filesystem.py

* Update filesystem.py

* Update os.py

* Start fixing tests

* Platform independent hooks

* Fix probe fd close

* Add broken_on_windows for integration tests

* Fix a lot of tests

* Use a python hook script, to prepare cross-platform

* New approach to be compliant with Linux and Windows on hook scripts

* New tests fixed

* Test for permissions on Windows

* Permissions comparison for Windows

* No broken tests in certbot core anymore

* Change mode

* Specific config for appveyor

* Use forked pebble for now

* Various fixes

* Assert file permissions for world on private keys

* Clean code

* Fix several things

* Add integration target

* Optimize integration env

* Re-enable all AppVeyor envs

* Use again official pebble

* Update pebble_artifacts.py

* Set PYTEST_ADDOPTS silently

* Update appveyor.yml

* Pin pywin32 for tests, give a minimal requirement for certbot.

* Remove injection of nginx in PATH

* Clean debug code

* Various cleanup, ensure to remove workspace after tests

* Update tox target

* Improve assertions. Control the keyword echoed in hooks

* Fix for virtualenv on Python 3.7.4 for Windows

* Update certbot-ci/certbot_integration_tests/certbot_tests/assertions.py

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Add conditionally pywin in certbot-ci like in certbot
2019-08-07 00:02:16 +02:00
Adrien Ferrand
36b4c312c6 Upgrade virtualenv in dev/tests environments (#7287)
AppVeyor recently upgrade the Python 3.7.x installed in their VM to 3.7.4. However, virtualenv 16.6.1 is broken on that specific version of Python for Windows.

This PR upgrade virtualenv installed for a dev/test environment from 16.6.1 to 16.6.2 in order to fix this issue, and repair the CI jobs execute by AppVeyor on PRs.
2019-08-02 09:47:36 -07:00
Brad Warren
f7c736da6f
Update pexpect to fix Python 3.7 dev venvs. (#7259) 2019-07-18 15:44:01 -07:00
Brad Warren
3e872627d8 Pin/upgrade virtualenv in our tests (#7211)
* Update virtualenv to the latest version.

* Use venv from pip and pin more packages.

* Pin codecov.

* update appveyor config

* Write the path separator backwards.

* s/pip_install.py install/pip_install.py

* Prefix tools\\pip_install.py with python exe.

* Upgrade py to fix AppVeyor failures.

* add back comment

* Update virtualenv with CERTBOT_NO_PIN.

* Pass -U to upgrade tox and deps.

* Upgrade virtualenv.
2019-07-02 20:02:00 +03:00
Adrien Ferrand
2b1c77c1ca [Unix] Create a framework for certbot integration tests: PART 2 (#6821)
* Second part: integration tests for certbot core

* Specific coverages

* Add comments

* Improve names

* Suspend fail-under until complete coverage

* Implement a minimal functional example

* Update certbot-ci/certbot_integration_tests/certbot_tests/conftest.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/certbot_tests/context.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/certbot_tests/context.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/utils/misc.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/utils/misc.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Fist set of corrections after review

* Fix test and test deploy hook flag

* Improve an assertion, remove conftest

* Add a test to cover all assertions. Remove the CSR logic for now

* Update certbot-ci/certbot_integration_tests/utils/misc.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/utils/misc.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/utils/misc.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/utils/misc.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Some corrections

* Add the http-01 test to complete coverage

* Add a comment.

* Make single requirements

* Update certbot-ci/certbot_integration_tests/certbot_tests/context.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Revert "Some corrections"

This reverts commit 6f20a060e5.

# Conflicts:
#	certbot-ci/certbot_integration_tests/certbot_tests/context.py
#	certbot-ci/certbot_integration_tests/certbot_tests/test_main.py

* Clean join

* Update certbot-ci/certbot_integration_tests/certbot_tests/context.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/certbot_tests/context.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Change assertion name

* Rewrite http auth hook as real python scripts

* Correct output in some OS

* Try a direct execution

* Fix shebang

* Correct a script

* Update certbot config

* Call explicitly with python, to be cross platform compatible

* Avoid infinite loops. Improve documentation.

* Fix syntax
2019-04-11 18:07:36 -07:00
Brad Warren
12ab59e1fc
Merge pull request #6932 from adferrand/pylint-squash
Update Pylint to 1.9.4 (squashed PR)
2019-04-09 10:47:19 -07:00
Adrien Ferrand
04152c21b5 Update to Pylint 1.9.4 and corrections 2019-04-09 09:22:19 +02:00
Brad Warren
c77159a30c
Update the lexicon version used in tests/Docker. (#6929)
This will resolve problems with certbot-dns-dnsimple in Docker.
2019-04-08 12:51:52 -07:00
Brad Warren
0c6d83dc39
Merge pull request #6786 from certbot/pin-deps_continued
This continues from the work of @sydneyli in PR #6671
I didn't do much here. Basically added support for reading data from sys.stdin to both tools/merge_requirements.py and tools/strip_hashes.py as well as support for reading files from paths passed as cli parameters to strip_hashes.py.

Reading the filepaths was not strictly required, but I thought would be a good thing to do in order to keep the tooling usage options consistent.

Fixes #6581

* Generate constraints file to pin deps in Docker images

Dockerfiles pin versions using constraints file

Pulling out strip_hashes and add --no-deps flag

* Add stdin option for merge_requirements

Add stdin and file path support to strip_hashes

* Address review comments
2019-03-04 09:02:53 -08:00
Adrien Ferrand
841f8efd0a [Unix] Create a framework for certbot integration tests: PART 1 (#6578)
* First part

* Several optimizations about the docker env setup

* Documentation

* Various corrections and documentation. Add acme and certbot explicitly as dependencies of certbot-ci.

* Correct a variable misinterpreted as a pytest hook

* Correct strict parsing option on pebble

* Refactor acme setup to be executed from pytest hooks.

* Pass TRAVIS env variable to trigger specific xdist logic

* Retrigger build.

* Work in progress

* Config operational

* Propagate to xdist

* Corrections on acme and misc

* Correct subnet for pebble

* Remove gobetween, as tls-sni challenges are not tested anymore.

* Improve pebble setup. Reduce LOC.

* Update acme.py

* Optimize acme ca setup, with less temporary assets

* Silent setup

* Clean code

* Remove unused workspace

* Use default network driver

* Remove bridge

* Update package documentation

* Remove rerun capability for integration tests, not needed.

* Add documentation

* Variable for all ports and subnets used by the stack

* Update certbot-ci/certbot_integration_tests/conftest.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/utils/acme.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/utils/misc.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update tox.ini

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/utils/misc.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/utils/acme.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/utils/acme.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Update certbot-ci/certbot_integration_tests/conftest.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Rename to acme_server

* Add comment

* Refactor in a unique context fixture

* Remove the need of CERTBOT_ACME_XDIST environment variable

* Remove nonstrict/strict options in pebble

* Clean dependencies

* Clean tox

* Change function name

* Add comment about coveragerc specificities

* Change a comment.

* Update setup.py

* Update conftest.py

* Use the production-ready docker-compose.yml file for Pebble

* New style class

* Tune pebble to have a stable test environment

* Pin a dependency
2019-03-01 13:18:06 -08:00
sydneyli
7c731599a0
Generate constraints file to pin deps in Docker images
Dockerfiles pin versions using constraints file

Pulling out strip_hashes and add --no-deps flag
2019-02-20 16:59:55 +02:00
Adrien Ferrand
bb1242c047 Process isolated local oldest requirements for DNS plugins (#6653)
* Include local-oldest-requirements in the constraints when oldest is invoked

* Add oldest constraints for lexicon

* Skip editable requirements during merge

* Pin to lexicon 2.2.1 for oldest requirements. Override locally oldest if needed for specific providers.
2019-01-17 15:21:13 -08:00
Adrien Ferrand
651de2dd2f Update Certbot dependency to Lexicon to 3.x (#6593)
This PR updates Lexicon dependency to the latest version available, 3.0.6, for every lexicon-based DNS plugins. It updates also the provider construction to use the new ConfigResolverobject, and to remove the legacy configuration process.
2019-01-10 11:36:15 -08:00
sydneyli
f5aad1440f Conditionally depend on imgconverter for newer versions of Sphinx (#6536)
Fixes #6343.

* conditionally depend on imgconverter

* Pin docutils dependency for old Sphinx bug
2018-12-04 10:56:15 -08:00
Erica Portnoy
5700af594b Merge branch 'master' into warnings-are-errors 2018-11-17 02:29:20 +00:00
Adrien Ferrand
e6e323e3ff Update Lexicon to correct use of HTTP proxy on OVH provider (#6479)
This PR update requirement of Lexicon to 2.7.14 on OVH plugin, to allow HTTP proxy to be used correctly when underlying OVH provider is invoked.

* Update Lexicon to correct use of HTTP proxy on OVH provider

* Update dev_constraints.txt

* Update CHANGELOG.md
2018-11-07 07:49:13 -08:00
Erica Portnoy
64a61fa6d4 Use a newer version of botocore that doesn't vendor requests, which we need because newer versions of requests don't have the DeprecationWarning 2018-11-02 14:53:55 -07:00
Adrien Ferrand
efd2ed1bdb Correct OVH integration tests on machines without internet access (#6380)
* Correct OVH integration tests on machines without internet.

* Update changelog
2018-09-18 17:35:28 -07:00
Brad Warren
d8057f0e17
Fix Sphinx (#6070)
Fixes #4686.

In Sphinx 1.6, they changed how they handle images in latex and PDF files. You can learn more about this by reading the linked issue (or I can answer any questions), but the shortish version is we now need to use the extension sphinx.ext.imgconverter. This is only available in Sphinx 1.6+.

I also updated our pinned versions to use the latest Sphinx and a new dependency it pulled in called sphinxcontrib-websupport. To build the latex and PDF docs, you must first run:

apt-get install imagemagick latexmk texlive texlive-latex-extra

Afterwards, if you create the normal Certbot dev environment using this branch, activate the virtual environment, and from the root of the repo run make -C docs clean latex latexpdf, you'll successfully build the PDF docs.

* fix #4686

* bump minimum Sphinx req
2018-08-06 09:45:56 -07:00
Yoan Blanc
6262921315 bump pytest-xdist to 1.22.5 (#6253)
Signed-off-by: Yoan Blanc <yoan.blanc@exoscale.ch>
2018-07-31 10:31:36 -07:00
Brad Warren
cdf93de338
Full Python 3.7 support (#6182)
Now that yaml/pyyaml#126 is resolved, #6170 can be reverted by bumping the pinned version of PyYAML.

You can see this code passing with full macOS and integration tests at https://travis-ci.org/certbot/certbot/builds/400957729.

* Revert "Allow py37 testing (#6170)"

This reverts commit cad95466b0.

* Bump pyyaml pinning to work on Python 3.7.
2018-07-09 09:16:44 -07:00
Brad Warren
dd600db436
Upgrade pinned josepy version (#6184)
We released josepy 1.1.0 a while ago to work around newer versions of cryptography deprecating some of the functionality we were using. We haven't yet upgraded our pinned josepy version though and since #6169 has landed, we're now seeing these deprecation warnings in our tests. This would be shown to certbot-auto users as well.

This PR removes these warnings by upgrading our pinned version of josepy.

* update pinned josepy version

* build leauto

* update pinned dev version of josepy
2018-07-09 09:16:08 -07:00
Brad Warren
3a8de6d172
Upgrade pinned twine version. (#6078)
For the past couple of releases, twine has errored while trying to upload
packages and this is fixed by upgrading to a newer version of twine. This
commit updates our pinned version installed when using tools/venv.sh to the
latest available version. pkginfo had to be upgraded as well to support the
latest version of twine.
2018-06-07 07:50:36 -07:00
Dmitry Figol
36dfd06503 Prepare certbot module for mypy check untyped defs (#6005)
* Prepare certbot module for mypy check untyped defs

* Fix #5952

* Bump mypy to version 0.600 and fix associated bugs

* Fix pylint bugs after introducing mypy

* Implement Brad's suggestions

* Reenabling pylint and adding nginx mypy back
2018-05-18 06:28:17 -07:00
ohemorange
2d31598484 Get mypy tox env running in the current setup (#5861)
* get mypy tox env running in the current setup

* use any python3 with mypy

* pin mypy dependencies
2018-04-12 15:47:39 -07:00
sydneyli
8231b1a19c Pin Lexicon version to 2.2.1 (#5803) 2018-03-29 17:09:21 -07:00
Brad Warren
39472f88de
reduce ipdb version (#5408) 2018-01-10 13:26:31 -08:00
Brad Warren
a1aea021e7 Pin dependencies in oldest tests (#5316)
* Add tools/merge_requirements.py

* Revert "Fix oldest tests by pinning Google DNS deps (#5000)"

This reverts commit f68fba2be2.

* Add tools/oldest_constraints.txt

* Remove oldest constraints from tox.ini

* Rename dev constraints file

* Update tools/pip_install.sh

* Update install_and_test.sh

* Fix pip_install.sh

* Don't cat when you can cp

* Add ng-httpsclient to dev constraints for oldest tests

* Bump tested setuptools version

* Update dev_constraints comment

* Better document oldest dependencies

* test against oldest versions we say we require

* Update dev constraints

* Properly handle empty lines

* Update constraints gen in pip_install

* Remove duplicated zope.component

* Reduce pyasn1-modules dependency

* Remove blank line

* pin back google-api-python-client

* pin back uritemplate

* pin josepy for oldest tests

* Undo changes to install_and_test.sh

* Update install_and_test.sh description

* use split instead of partition
2017-12-18 12:31:36 -08:00
Renamed from tools/pip_constraints.txt (Browse further)