certbot/tests/letstest/scripts/test_apache2.sh

138 lines
5 KiB
Bash
Raw Permalink Normal View History

#!/bin/bash -x
# $OS_TYPE $PUBLIC_IP $PRIVATE_IP $PUBLIC_HOSTNAME $BOULDER_URL
# are dynamically set at execution
if [ "$OS_TYPE" = "ubuntu" ]
then
CONFFILE=/etc/apache2/sites-available/000-default.conf
sudo apt-get update
sudo apt-get -y --no-upgrade install apache2 curl
sudo apt-get -y install realpath # needed for test-apache-conf
# For apache 2.4, set up ServerName
sudo sed -i '/ServerName/ s/#ServerName/ServerName/' $CONFFILE
sudo sed -i '/ServerName/ s/www.example.com/'$PUBLIC_HOSTNAME'/' $CONFFILE
if [ $(python3 -V 2>&1 | cut -d" " -f 2 | cut -d. -f1,2 | sed 's/\.//') -lt 36 ]
Update test farm tests to stop using certbot-auto (#8207) * Create bootstrap script * Delete a whole bunch of the bootstrap script * modify test_tests to use new script * put python version checking in back in * add x * call the venv creation from inside the bootstrap * add targets back * modify test_apache2 to use new format * shouldn't need virtualenv on rhel * readd targets * Update test_sdists to use new script * move setting up venv back out of script so it's not run with sudo * take venv3.py call out of bootstrap in all scripts * add additional python3-devel pkg name * fix test_sdists * enable additional rhel7 repos * clean up code and comments * Update tests and instructions to use auto_targets.yaml with test_leauto_upgrades.sh and test_letsencrypt_auto_certonly_standalone.sh * only install python3-devel.x86_64 for rhel7 * Upgrade python version for debian in test_apache2.sh * don't run test_tests or test_sdists on debian 9 or ubuntu 16.04 * Add 20.04 and 20.04 arm images to targets.yaml * use pyenv to upgrade to python3.5 * remove arm64 instance because it's having auth trouble * correct pyenv usage on ubuntu * add arm64 target to targets.yaml * replace debian 9 arm64 with ubuntu 20 * don't try to upgrade a perfectly good python version * let's just add ubuntu20 to apache2_targets while we're here * uncomment test_apache2 * move adding python3-devel.x86_64 to bootstrap_os_packages to avoid potential race condition * no need to specify the arch once extra rhel7 repos enabled * explicitly specify python3 * don't fail if we can't enable rhel7 extras * capture python36-devel as well
2020-08-18 13:07:27 -04:00
then
# Upgrade python version using pyenv because py3.5 is deprecated
# Don't upgrade if it's already 3.8 because pyenv doesn't work great on arm, and
# our arm representative happens to be ubuntu20, which already has a perfectly
# good version of python.
sudo apt-get install -y make gcc build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev libffi-dev liblzma-dev python-openssl git # pyenv deps
curl https://pyenv.run | bash
export PATH="~/.pyenv/bin:$PATH"
pyenv init -
pyenv virtualenv-init -
pyenv install 3.8.5
pyenv global 3.8.5
# you do, in fact need to run these again, exactly like this.
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
fi
elif [ "$OS_TYPE" = "centos" ]
then
CONFFILE=/etc/httpd/conf/httpd.conf
sudo setenforce 0 || true #disable selinux
sudo yum -y install httpd
2016-10-04 19:45:24 -04:00
sudo yum -y install nghttp2 || echo this is probably ok but see https://bugzilla.redhat.com/show_bug.cgi?id=1358875
sudo service httpd start
sudo mkdir -p /var/www/$PUBLIC_HOSTNAME/public_html
sudo chmod -R oug+rwx /var/www
sudo chmod -R oug+rw /etc/httpd
sudo echo '<html><head><title>foo</title></head><body>bar</body></html>' > /var/www/$PUBLIC_HOSTNAME/public_html/index.html
2016-04-14 20:10:27 -04:00
sudo mkdir /etc/httpd/sites-available #certbot requires this...
sudo mkdir /etc/httpd/sites-enabled #certbot requires this...
#sudo echo "IncludeOptional sites-enabled/*.conf" >> /etc/httpd/conf/httpd.conf
sudo echo """
<VirtualHost *:80>
ServerName $PUBLIC_HOSTNAME
DocumentRoot /var/www/$PUBLIC_HOSTNAME/public_html
ErrorLog /var/www/$PUBLIC_HOSTNAME/error.log
CustomLog /var/www/$PUBLIC_HOSTNAME/requests.log combined
</VirtualHost>""" >> /etc/httpd/conf.d/$PUBLIC_HOSTNAME.conf
#sudo cp /etc/httpd/sites-available/$PUBLIC_HOSTNAME.conf /etc/httpd/sites-enabled/
fi
2016-04-14 20:10:27 -04:00
# Run certbot-apache2.
cd letsencrypt
echo "Bootstrapping dependencies..."
Update test farm tests to stop using certbot-auto (#8207) * Create bootstrap script * Delete a whole bunch of the bootstrap script * modify test_tests to use new script * put python version checking in back in * add x * call the venv creation from inside the bootstrap * add targets back * modify test_apache2 to use new format * shouldn't need virtualenv on rhel * readd targets * Update test_sdists to use new script * move setting up venv back out of script so it's not run with sudo * take venv3.py call out of bootstrap in all scripts * add additional python3-devel pkg name * fix test_sdists * enable additional rhel7 repos * clean up code and comments * Update tests and instructions to use auto_targets.yaml with test_leauto_upgrades.sh and test_letsencrypt_auto_certonly_standalone.sh * only install python3-devel.x86_64 for rhel7 * Upgrade python version for debian in test_apache2.sh * don't run test_tests or test_sdists on debian 9 or ubuntu 16.04 * Add 20.04 and 20.04 arm images to targets.yaml * use pyenv to upgrade to python3.5 * remove arm64 instance because it's having auth trouble * correct pyenv usage on ubuntu * add arm64 target to targets.yaml * replace debian 9 arm64 with ubuntu 20 * don't try to upgrade a perfectly good python version * let's just add ubuntu20 to apache2_targets while we're here * uncomment test_apache2 * move adding python3-devel.x86_64 to bootstrap_os_packages to avoid potential race condition * no need to specify the arch once extra rhel7 repos enabled * explicitly specify python3 * don't fail if we can't enable rhel7 extras * capture python36-devel as well
2020-08-18 13:07:27 -04:00
sudo tests/letstest/scripts/bootstrap_os_packages.sh
if [ $? -ne 0 ] ; then
exit 1
fi
tools/venv.py -e acme[dev] -e certbot[dev,docs] -e certbot-apache -e certbot-ci
PEBBLE_LOGS="acme_server.log"
PEBBLE_URL="https://localhost:14000/dir"
# We configure Pebble to use port 80 for http-01 validation rather than an
# alternate port because:
# 1) It allows us to test with Apache configurations that are more realistic
# and closer to the default configuration on various OSes.
# 2) As of writing this, Certbot's Apache plugin requires there to be an
# existing virtual host for the port used for http-01 validation.
venv/bin/run_acme_server --http-01-port 80 > "${PEBBLE_LOGS}" 2>&1 &
Update Fedora AMI (#7102) Fixes #6955. This updates the Fedora version used in our test farm tests to Fedora 30. The AMI ID comes from https://alt.fedoraproject.org/cloud/ where it is listed as their standard HVM AMI for the region we use us-east-1 (US East (N. Virginia)). Unfortunately, there were a lot of small changes required for this. The big reason for this is on Fedora, there isn't a Python 2 executable installed. In fact, there's not even an executable named python. It's just python3. Rather than installing another Python in each test, I wrote a script that the test scripts can share to figure out the different paths and names that should be used in their script. (This isn't used in test_sdists.sh because the logic is a little different.) Other changes here worth flagging are: I changed the name of the variable RUN_PYTHON3_TESTS in test_leauto_upgrades.sh to RUN_RHEL6_TESTS. The tests that are run when this variable is set test the upgrade from Python 2 to Python 3 on RHEL 6. I think this new name is much better now that we also have Fedora running Python 3. I made tools/simple_http_server.py work on Python 3. You can see tests passing with these changes at https://travis-ci.com/certbot/certbot/builds/113821476. I also ran test_tests.sh and they passed. * Update to Fedora 30 in test farm tests. Fedora 28 is likely to reach its EOL soon. * Add set_python_envvars.sh. * Fix test_apache2.sh on python3 only distros. * Fix test_leauto_upgrades.sh on python3 systems. * Fix certonly_standalone tests with python3 only * Fix test_sdists.sh on python3 only distros. * Make simple_http_server.py work on Python 3. * add comments
2019-05-31 21:08:52 -04:00
DumpPebbleLogs() {
if [ -f "${PEBBLE_LOGS}" ] ; then
echo "Pebble's logs were:"
cat "${PEBBLE_LOGS}"
fi
}
for n in $(seq 1 150) ; do
if curl --insecure "${PEBBLE_URL}" 2>/dev/null; then
break
else
echo "waiting for pebble"
sleep 1
fi
done
if ! curl --insecure "${PEBBLE_URL}" 2>/dev/null; then
echo "timed out waiting for pebble to start"
DumpPebbleLogs
exit 1
fi
sudo "venv/bin/certbot" -v --debug --text --agree-tos --no-verify-ssl \
--renew-by-default --redirect --register-unsafely-without-email \
--domain "${PUBLIC_HOSTNAME}" --server "${PEBBLE_URL}"
if [ $? -ne 0 ] ; then
FAIL=1
fi
Disable TLS session tickets in Apache (#7771) Fixes #7350. This PR changes the parsed modules from a `set` to a `dict`, with the filepath argument as the value. Accordingly, after calling `enable_mod` to enable `ssl_module`, modules now need to be re-parsed, so call `reset_modules`. * Add mechanism for selecting apache config file, based on work done in #7191. * Check OpenSSL version * Remove os imports * debian override still needs os * Reformat remaining apache tests with modules dict syntax * Clean up more apache tests * Switch from property to method for openssl and add tests for coverage. * Sometimes the dict location will be None in which case we should in fact return None * warn thoroughly and consistently in openssl_version function * update tests for new warnings * read file as bytes, and factor out the open for testing * normalize ssl_module_location path to account for being relative to server root * Use byte literals in a python 2 and 3 compatible way * string does need to be a literal * patch builtins open * add debug, remove space * Add test to check if OpenSSL detection is working on different systems * fix relative test location for cwd * put </IfModule> on its own line in test case * Revert test file to status in master. * Call augeas load before reparsing modules to pick up the changes * fix grep, tail, and mod_ssl location on centos * strip the trailing whitespace from fedora * just use LooseVersion in test * call apache2ctl on debian systems * Use sudo for apache2ctl command * add check to make sure we're getting a version * Add boolean so we don't warn on debian/ubuntu before trying to enable mod_ssl * Reduce warnings while testing by setting mock _openssl_version. * Make sure we're not throwing away any unwritten changes to the config * test last warning case for coverage * text changes for clarity
2020-03-23 19:49:52 -04:00
# Check that ssl_module detection is working on various systems
if [ "$OS_TYPE" = "ubuntu" ] ; then
MOD_SSL_LOCATION="/usr/lib/apache2/modules/mod_ssl.so"
APACHE_NAME=apache2ctl
elif [ "$OS_TYPE" = "centos" ]; then
MOD_SSL_LOCATION="/etc/httpd/modules/mod_ssl.so"
APACHE_NAME=httpd
fi
OPENSSL_VERSION=$(strings "$MOD_SSL_LOCATION" | egrep -o -m1 '^OpenSSL ([0-9]\.[^ ]+) ' | tail -c +9)
APACHE_VERSION=$(sudo $APACHE_NAME -v | egrep -o 'Apache/([0-9]\.[^ ]+)' | tail -c +8)
"venv/bin/python" tests/letstest/scripts/test_openssl_version.py "$OPENSSL_VERSION" "$APACHE_VERSION"
Disable TLS session tickets in Apache (#7771) Fixes #7350. This PR changes the parsed modules from a `set` to a `dict`, with the filepath argument as the value. Accordingly, after calling `enable_mod` to enable `ssl_module`, modules now need to be re-parsed, so call `reset_modules`. * Add mechanism for selecting apache config file, based on work done in #7191. * Check OpenSSL version * Remove os imports * debian override still needs os * Reformat remaining apache tests with modules dict syntax * Clean up more apache tests * Switch from property to method for openssl and add tests for coverage. * Sometimes the dict location will be None in which case we should in fact return None * warn thoroughly and consistently in openssl_version function * update tests for new warnings * read file as bytes, and factor out the open for testing * normalize ssl_module_location path to account for being relative to server root * Use byte literals in a python 2 and 3 compatible way * string does need to be a literal * patch builtins open * add debug, remove space * Add test to check if OpenSSL detection is working on different systems * fix relative test location for cwd * put </IfModule> on its own line in test case * Revert test file to status in master. * Call augeas load before reparsing modules to pick up the changes * fix grep, tail, and mod_ssl location on centos * strip the trailing whitespace from fedora * just use LooseVersion in test * call apache2ctl on debian systems * Use sudo for apache2ctl command * add check to make sure we're getting a version * Add boolean so we don't warn on debian/ubuntu before trying to enable mod_ssl * Reduce warnings while testing by setting mock _openssl_version. * Make sure we're not throwing away any unwritten changes to the config * test last warning case for coverage * text changes for clarity
2020-03-23 19:49:52 -04:00
if [ $? -ne 0 ] ; then
FAIL=1
fi
if [ "$OS_TYPE" = "ubuntu" ] ; then
export SERVER="${PEBBLE_URL}"
"venv/bin/tox" -e apacheconftest
else
echo Not running hackish apache tests on $OS_TYPE
fi
if [ $? -ne 0 ] ; then
FAIL=1
fi
# return error if any of the subtests failed
if [ "$FAIL" = 1 ] ; then
DumpPebbleLogs
exit 1
fi