Andrzej Górski
9da07590bd
Remove --fast from the test farm tests ( #7427 )
2019-10-08 21:24:55 +02:00
Brad Warren
bf9c681c4f
fix backwards logic ( #7265 )
2019-07-25 10:20:52 +02:00
Brad Warren
a37a4486cf
Add Debian ARM AMI. ( #7189 )
...
Inspired by the number of ARM users we have (and because I want to rip out the only 32 bit test we have which without this PR would remove all tests we have on non-x86_64 architectures), this test adds an ARM image to the test farm tests. The image ID was taken from https://wiki.debian.org/Cloud/AmazonEC2Image/Stretch , you can see tests passing at https://travis-ci.com/certbot/certbot/builds/116857897 , and I ran test_tests.sh locally and it passed.
2019-06-25 14:03:45 -07:00
Brad Warren
18797dca79
Remove scripts that are never run. ( #7111 )
...
* Remove scripts that are never run.
* Update example in multitester.py docstring.
2019-06-03 10:20:20 +03:00
Brad Warren
d1753e46f9
Allow magic profile name none for configuring test farm tests. ( #7069 )
...
* Allow magic profile name none.
* Naming is hard.
2019-05-17 20:40:20 +02:00
Brad Warren
16834a0d78
Stop sharing state between processes in test farm tests ( #7057 )
...
* Set LOGDIR at top of script.
* Set sentinel at top of script.
* Don't use EC2 global to block on instance start.
* Remove global boto3 state.
* Pass in boulder_url.
* Create main function.
* Add link to reload docs.
2019-05-17 20:36:58 +02:00
Brad Warren
11c3e7107c
Exit with a nonzero status when tests fail. ( #7065 )
2019-05-17 19:44:51 +02:00
Brad Warren
6bcd0415e0
Fix race condition adding tags to instance. ( #7080 )
2019-05-17 11:27:40 +02:00
Brad Warren
7a6545b747
Regularly print output in the test farm tests ( #7079 )
...
* Occasionally print output in test farm tests.
* Flush output.
2019-05-17 11:21:57 +02:00
Brad Warren
f4bbaadd18
Fix cleanup on failure. ( #7070 )
...
* Fix cleanup on failure.
* Incrementally build instances list.
2019-05-17 11:17:27 +02:00
ohemorange
f0f5bb4fc0
Update test farm version of boulder to current master ( #7002 )
...
Recent changes are no longer compatible with the old version of boulder used in the test farm tests. This PR updates the version of boulder used, and runs it with the new way of running boulder.
A new ami was created and is used here that uses Ubuntu 18.04, so that docker-compose can be installed more properly.
Removed commented-out section about rabbitmq that was already deprecated.
Switched to using the public DNS resolver 8.8.8.8 for the tests because the way to find the correct local resolver changed.
2019-04-30 13:13:37 -07:00
ohemorange
eef4c47633
Add failure message if test farm tests do not run the correct number of tests. ( #6771 )
...
Fixes #6748 .
2019-02-20 15:20:44 -08:00
Brad Warren
8e5b2ac5b5
Stop multitester.py from eating errors. ( #6705 )
2019-01-25 23:24:11 +01:00
Brad Warren
cd2edeff1b
Fix test farm tests ( #6335 )
...
* update CentOS AMI ids
* Remove assumption of usable default subnet
2018-09-05 13:12:05 -07:00
Brad Warren
783b6e4746
Automate EBS cleanup ( #6160 )
...
* ensure volume cleanup
* remove volume cleanup
* cleanup function and output
2018-07-17 17:19:04 -07:00
cclauss
09b5927e6a
from botocore.exceptions import ClientError ( #5507 )
...
Fixes undefined name 'botocore' in flake8 testing of https://github.com/certbot/certbot
$ __flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics__
```
./tests/letstest/multitester.py:144:12: F821 undefined name 'botocore'
except botocore.exceptions.ClientError as e:
^
1 F821 undefined name 'botocore'
```
2018-02-15 11:07:35 -08:00
Brad Warren
3780d068d1
Fix test farm tests
2016-04-19 14:11:17 -04:00
Noah Swartz
842b609dad
Merge pull request #2841 from letsencrypt/module-rename
...
Module rename
2016-04-14 17:35:09 -07:00
Brad Warren
e353f8fabc
letstest is a Certbot integration test tool
2016-04-14 17:16:48 -07:00
Brad Warren
b4f6ed8470
rename letstest stuff
2016-04-14 17:10:27 -07:00
Brad Warren
7228c0c9ed
'co' is not a git command
2016-04-14 08:57:34 -07:00
Peter Eckersley
f4c33656a2
Try to finally prevent dangling AWS volumes
2016-02-23 12:08:01 -08:00
Peter Eckersley
1f31cf1a30
Quick test farm fix
2016-02-10 19:09:05 -08:00
Roy Wellington Ⅳ
9c28364477
Make exception syntax Python 3 compatible.
...
Translate all except and raise statements that are in the old form to
the Python 3 compatible format.
2016-01-30 19:53:50 -08:00
Peter Eckersley
b8281fdd34
Retry in case of ephemeral errors
2016-01-20 14:25:01 -08:00
Peter Eckersley
3e1bc19e0f
Optional flag for faster AMIs
2015-12-25 10:43:52 -08:00
Peter Eckersley
e0837ad41f
[letstest] create & reuse a persistent boulder server
...
Reduces minimum multitester.py runtime to just a bit under 10 minutes
2015-12-25 02:39:48 -08:00
Peter Eckersley
49721c7d01
Merge remote-tracking branch 'letstest/upgrade-test' into testfarm
2015-12-23 15:14:30 -08:00
Peter Eckersley
87dfe8c2b2
Move everything into tests/letstest
2015-12-21 11:12:01 -08:00
