Commit graph

355 commits

Author SHA1 Message Date
Brad Warren
1ea5bb5fb8 Merge branch 'louder' into test-letstest-all-changes2 2019-05-16 11:41:43 -07:00
Brad Warren
ebeb7dd8ce flush output 2019-05-16 11:40:42 -07:00
Brad Warren
c6537a3d6f Merge branch 'louder' into test-letstest-all-changes2 2019-05-16 10:31:28 -07:00
Brad Warren
0e9cadcafd Occasionally print output in test farm tests. 2019-05-16 10:24:23 -07:00
Brad Warren
d203947757 Merge remote-tracking branch 'origin/travis-test-farm' into test-letstest-all-changes2 2019-05-15 20:18:37 -07:00
Brad Warren
bd68c00ac7 Merge branch 'dont-suppress-failure' into test-letstest-all-changes2 2019-05-15 20:18:30 -07:00
Brad Warren
1d548c80a6 Merge remote-tracking branch 'origin/letstest-no-profile' into test-letstest-all-changes2 2019-05-15 20:16:36 -07:00
Brad Warren
9044f6c975 Merge branch 'fix-centos6-test-sdists' into test-letstest-all-changes2 2019-05-15 20:15:40 -07:00
Brad Warren
03110c0857 Merge branch 'apache2_targets' into test-letstest-all-changes2 2019-05-15 20:15:30 -07:00
Brad Warren
d11fa0e2d7 Merge branch 'letstest-exit-status' into test-letstest-all-changes2 2019-05-15 20:15:24 -07:00
Brad Warren
7ea7638f29 Merge branch 'letstest-requirements' into test-letstest-all-changes2 2019-05-15 20:14:01 -07:00
Brad Warren
308157ed06 Merge branch 'no-sharing' into test-letstest-all-changes2 2019-05-15 20:13:53 -07:00
Brad Warren
8d898a7183 Incrementally build instances list. 2019-05-15 20:11:46 -07:00
Brad Warren
91b36e3b7f Fix cleanup on failure. 2019-05-15 18:45:10 -07:00
Brad Warren
afd823c7ec Make script executable 2019-05-15 18:22:57 -07:00
Brad Warren
500ae4b85e Move test commands to external script. 2019-05-15 18:22:10 -07:00
Brad Warren
b9d71190a7 Allow magic profile name none. 2019-05-15 16:56:06 -07:00
Brad Warren
a76020f6bc Add encrypted private key. 2019-05-15 15:44:25 -07:00
Brad Warren
846de3cc65 fix venv path 2019-05-15 15:16:15 -07:00
Brad Warren
10fa95cba8 Use Python 3 when appropriate. 2019-05-15 15:09:33 -07:00
Brad Warren
60fba5fcfe Update known good apache2 targets. 2019-05-15 15:03:10 -07:00
Brad Warren
ff7fc8486a Exit with a nonzero status when tests fail. 2019-05-14 16:12:09 -07:00
Adrien Ferrand
9a7f774706 [Unix] Create a framework for certbot integration tests: PART 5-FINAL (#6989)
* Connect certbot-ci to travis. Remove old bash files.

* Configure test-everything

* Protect against import error

* Remove unused ignore

* Better handling of urllib3

* Correct path

* Remove a warning

* Correct call

* Protect atexit register execution

* Update docs/contributing.rst

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update docs/contributing.rst

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Add again some bash scripts to avoid breaking to much retro-compatiblity on third party scripts

* Move boulder-v1 and boulder-v2 in nightly tests

* Separate oldest unit tests and oldest integration tests

* Remove try/except

* Test integration included in toxenv

* Add a wait to avoid a transient issue on OCSP status in oldest tests

* Clean travis.yml, split other tests

* Remove useless config

* Update .travis.yml

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update tox.ini

* Update tox.ini

* Remove pytest-sugar

* Remove empty pytest.ini, tests are working without it
2019-05-14 13:56:32 -07:00
Brad Warren
bcfb070a9d update readme 2019-05-13 15:55:58 -07:00
Brad Warren
1ad6857b56 Add requirements.txt. 2019-05-13 15:55:33 -07:00
Brad Warren
3c041f1655 Create main function. 2019-05-09 16:43:40 -07:00
Brad Warren
a7fd7cdd87 Pass in boulder_url. 2019-05-09 16:43:09 -07:00
Brad Warren
4f510706a1 Remove global boto3 state. 2019-05-09 16:41:02 -07:00
Brad Warren
7f75454675 Don't use EC2 global to block on instance start. 2019-05-09 16:33:30 -07:00
Brad Warren
3fbdef6a64 Set sentinel at top of script. 2019-05-09 16:31:26 -07:00
Brad Warren
2065775193 Set LOGDIR at top of script. 2019-05-09 16:30:56 -07:00
Brad Warren
a754a90940 Fix test_leauto_upgrades.sh on CentOS 6. (#7037) 2019-05-06 16:50:03 -07:00
Brad Warren
e15e848474
Stop certbot-auto from printing blank lines (#7016)
Fixes #7012.

Apparently, the previous test we had here doesn't catch the case when certbot-auto prints blank lines. (I don't yet understand why so if someone does, please let me know!)

Regardless, I fixed up the test and verified it fails with the version of letsencrypt-auto in master and then fixed letsencrypt-auto so the test passes.

I ran test farm tests on the changes here and they passed on all instances.

* correct test

* fixes #7012
2019-05-02 11:36:47 -07:00
Brad Warren
862577fffc
Bump initial version to 0.33.1. (#7017)
We made this change locally yesterday while preparing the release.

I tested this change on all AMIs currently in the test farm as well as Fedora 29 and this test passed on all instances.
2019-05-02 11:32:49 -07:00
ohemorange
3900e56b52 Update Debian Jessie AMI to continue being able to use apt (#7003)
Fixes #6907.
2019-04-30 13:16:47 -07:00
ohemorange
f0f5bb4fc0 Update test farm version of boulder to current master (#7002)
Recent changes are no longer compatible with the old version of boulder used in the test farm tests. This PR updates the version of boulder used, and runs it with the new way of running boulder.

A new ami was created and is used here that uses Ubuntu 18.04, so that docker-compose can be installed more properly.

Removed commented-out section about rabbitmq that was already deprecated.

Switched to using the public DNS resolver 8.8.8.8 for the tests because the way to find the correct local resolver changed.
2019-04-30 13:13:37 -07:00
Brad Warren
d1330efe41
Print warning when certbot-auto has insecure permissions. (#6995)
This PR attempts to better inform people about the problem identified at https://community.letsencrypt.org/t/certbot-auto-deployment-best-practices/91979/.

I was hesitant to add the flag --no-permissions-check, however, if there's some obscure distro out there (or custom user setup) that has a strange users and groups, I didn't want us to either:

Have to put out a bug fix release
Refuse to fix the problem and let them deal with warnings on every run

* add check_permissions.py

* Update letsencrypt-auto.template.

* build letsencrypt-auto

* Add test_permissions_warnings to auto_test

* Allow uid/gid < 1000.

* Add --no-permissions-check to Certbot.

* Add --no-permissions-check to certbot-auto.

* Add test farm test that letsencrypt-auto is quiet.

As a bonus, this new test will catch problems like the one that the caused
0.33.1 point release.

* Update CHANGELOG about permissions check.

* Update permissions comment.

* Fix symlink handling.

* Use a better default in auto_test.py.
2019-04-30 10:45:03 -07:00
Brad Warren
d7610c1ae7 Update Fedora AMI (#6956)
* Update Fedora AMI to Fedora 28.

* Update initial version in test_leauto_upgrades.
2019-04-12 23:44:43 +02:00
Brad Warren
b0285438cc Move venv symlink check out of leauto_upgrades. (#6830)
* Move venv symlink check out of leauto_upgrades.

* Add back double venv check.
2019-04-10 18:24:32 -07:00
Brad Warren
944d0e05c8
Use venv over virtualenv in venv3 (#6922)
Fixes #6861.

_venv_common.py is no longer executable. The reason for this is the venv creation logic is now different between Python 2 and Python 3. We could add code that branches on the Python version running the script, but I personally think that's unnecessary.

--setuptools and --no-site-packages is no longer passed to virtualenv either. These flags were made noops in virtualenv 1.10 and 1.7 respectively, but all of CentOS 6, 7, Debian 8+, and Ubuntu 14.04+ have new enough versions of virtualenv where these flags are no longer necessary. They are not even accepted as flags to Python 3's venv module.

Use of VENV_ARGS from test_sdists.sh was also removed because that environment variable hasn't done anything in a while.

I ran test farm tests on test_apache2.sh and test_sdists.sh with these changes and they passed.

* Fixes #6861.

* _venv_common is no longer executable.
2019-04-05 15:01:09 -07:00
Brad Warren
aec29c2f1d Remove amazon linux test farm targets. (#6822) 2019-04-05 13:39:39 -07:00
Adrien Ferrand
821bec6997 Remove tls-sni related flags in cli. Add a deprecation warning instead. (#6853)
This PR is a part of the tls-sni-01 removal plan described in #6849.

This PR removes --tls-sni-01-port, --tls-sni-01-address and tls-sni-01/tls-sni options from --preferred-challenges. They are replace by deprecation warning, indicating that these options will be removed soon.

This deprecation, instead of complete removal, is done to avoid certbot instances to hard fail if some automated scripts still use these flags for some users.

Once this PR lands, we can remove completely theses flags in one or two release.

* Remove tls-sni related flags in cli. Add a deprecation warning instead.

* Adapt tests to cli and renewal towards tls-sni flags deprecation

* Add https_port option. Make tls_sni_01_port show a deprecation warning, but silently modify https_port if set

* Migrate last items

* Fix lint

* Update certbot/cli.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Ensure to remove all occurences of tls-sni-01

* Remove unused parameter

* Revert modifications on cli-help.txt

* Use logger.warning instead of sys.stderr

* Update the logger warning message

* Remove standalone_supported_challenges option.

* Fix order of preferred-challenges

* Remove supported_challenges property

* Fix some tests

* Fix lint

* Fix tests

* Add a changelog

* Clean code, fix test

* Update CI

* Reload

* No hard date for tls-sni removal

* Remove useless cast to list

* Update certbot/tests/renewal_test.py

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Add entry to the changelog

* Add entry to the changelog
2019-03-26 17:46:32 -07:00
Adrien Ferrand
d9880721b3 Remove tls sni in nginx plugin (#6857)
* Remove tls-sni from nginx config

* Add a dedicated configuration to define what is the HTTPS port for this certbot instance.

* Correct some tests

* Reestablish default vhost creation

* Clean tls references for nginx integration tests

* Associate https_port only to tests and nginx
2019-03-18 10:22:19 -07:00
Adrien Ferrand
acc918eee7 Remove tls-sni integration tests (#6852)
This PR is a part of the tls-sni-01 removal plan described in #6849.

This PR removes the tls-sni-01 challenge tests during the integration tests. The approach I used here is not to remove completely the existing test code, but simply editing it to use a http-01 challenge. Indeed:
* the current integration tests are strongly coupled, and would require more modifications that it is worth, because ...
* the certbot-ci project, that has already no tls-sni tests, will soon replace completely the current integration tests code.
2019-03-13 15:42:07 -07:00
Adrien Ferrand
cf29e89366 Move coverage computation during certbot integration tests at the end of the script (#6842)
Currently coverage invocation during integration tests on certbot core is misplaced, just before the OCSP statuses tests.

This PR move back the coverage invocation at the end of the script.
2019-03-11 16:16:48 -07:00
Adrien Ferrand
7161e792e8 Fix the Nginx configuration during integration tests (#6801)
If you execute `tests/lock_test.py` or `tox -e integration` on a fairly recent machine, you will get the following error during tests executing against a live Nginx instance:
```
no "ssl_certificate" is defined in server listening on SSL port while SSL handshaking, client: x.x.x.x, server: y:y:y:y:z
```

Indeed, having no defined ssl certificate for a ssl port would inevitably lead to an error during the handshake SSL process between a client and this mis-configured nginx instance.

However it was not a problem one year before, because the handshake was not occurring in practice: the test just need to have a nginx started, and then immediately proceed to modify the configuration with a correct SSL setup. And nginx was able to start with a mis-configuration on SSL. 

But then this fix has been done: https://trac.nginx.org/nginx/ticket/178

Basically with this, validation of the configuration is done during nginx startup, that will refuse to start with invalid configuration on SSL. Consequently, all related tests are failing with a sufficiently up-to-date nginx. For now, it is not seen on Travis because Ubuntu Trusty is used, with an old Nginx.

The PR fixes that, by generating on the fly self-signed certificates in the two impacted tests, and pushing the right parameters in the Nginx configuration.

* Fix nginx configuration with self-signed certificates generated on the fly

* Fix lint/mypy

* Fix old cryptography

* Unattended openssl

* Update lock_test.py
2019-03-01 13:54:09 -08:00
Adrien Ferrand
9c405a3cd1
Fix cryptography OCSP support (#6751)
* Reenabling OCSP cryptography support

* Refactor the validation logic of OCSP response to match the OpenSSL one

* Prepare runtime for OCSP response test

* Move unrelated test to another relevant place

* Reimplement OCSP status checks in integration tests

* Clean script

* Protect OCSP check against connection errors

* Update tests/certbot-boulder-integration.sh

Co-Authored-By: adferrand <adferrand@users.noreply.github.com>

* Cleaning

* Add a specific script for letsencrypt-auto install+help

* Remove inconsistent assertion

* Add executable permissions

* Remove unused variable

* Move testdata

* Corrected cleanup code

* Empty commit
2019-02-28 00:16:52 +01:00
Brad Warren
f105aedc92 Remove display.py. 2019-02-22 16:55:50 -08:00
ohemorange
eef4c47633 Add failure message if test farm tests do not run the correct number of tests. (#6771)
Fixes #6748.
2019-02-20 15:20:44 -08:00
Brad Warren
583d40f5cf
Pin pytest in test_sdists.sh. (#6764)
* pin pytest in test_sdists.sh.

* Use pip_install.py in test_tests.sh.
2019-02-14 15:26:44 -08:00