certbot/certbot-ci/windows_installer_integration_tests/test_main.py
Adrien Ferrand 7b35abbcb4
Windows installer integration tests (#7724)
As discussed in #7539, we need proper tests of the Windows installer itself in order to variety that all the logic contained in a production-grade runtime of Certbot on Windows is correctly setup by each version of the installer, and so for a variety of Windows OSes. 

This PR handles this requirement. The new `windows_installer_integration_tests` module in `certbot-ci` will:
* run the given Windows installer
* check that Certbot is properly installed and working
* check that the scheduled renew task is set up
* check that the scheduled task actually launch the Certbot renew logic

The Windows nightly tests are updated accordingly, in order to have the tests run on Windows Server 2012R2, 2016 and 2019.

These tests will evolve as we add more logic on the installer. 

* Configure an integration test testing the windows installer

* Write the test module

* Configurable installer path, prepare azure pipelines

* Fix option

* Update test_main.py

* Add confirmation for this destructive test

* Use regex to validate certbot --version output

* Explicit dependency on a log output

* Use an exception to ask confirmation

* Use --allow-persistent-changes
2020-02-05 14:12:29 -08:00

61 lines
2.5 KiB
Python

import os
import time
import unittest
import subprocess
import re
@unittest.skipIf(os.name != 'nt', reason='Windows installer tests must be run on Windows.')
def test_it(request):
try:
subprocess.check_call(['certbot', '--version'])
except (subprocess.CalledProcessError, OSError):
pass
else:
raise AssertionError('Expect certbot to not be available in the PATH.')
try:
# Install certbot
subprocess.check_call([request.config.option.installer_path, '/S'])
# Assert certbot is installed and runnable
output = subprocess.check_output(['certbot', '--version'], universal_newlines=True)
assert re.match(r'^certbot \d+\.\d+\.\d+.*$', output), 'Flag --version does not output a version.'
# Assert renew task is installed and ready
output = _ps('(Get-ScheduledTask -TaskName "Certbot Renew Task").State', capture_stdout=True)
assert output.strip() == 'Ready'
# Assert renew task is working
now = time.time()
_ps('Start-ScheduledTask -TaskName "Certbot Renew Task"')
status = 'Running'
while status != 'Ready':
status = _ps('(Get-ScheduledTask -TaskName "Certbot Renew Task").State', capture_stdout=True).strip()
time.sleep(1)
log_path = os.path.join('C:\\', 'Certbot', 'log', 'letsencrypt.log')
modification_time = os.path.getmtime(log_path)
assert now < modification_time, 'Certbot log file has not been modified by the renew task.'
with open(log_path) as file_h:
data = file_h.read()
assert 'no renewal failures' in data, 'Renew task did not execute properly.'
finally:
# Sadly this command cannot work in non interactive mode: uninstaller will ask explicitly permission in an UAC prompt
# print('Uninstalling Certbot ...')
# uninstall_path = _ps('(gci "HKLM:\\SOFTWARE\\Wow6432Node\\Microsoft\\Windows\\CurrentVersion\\Uninstall"'
# ' | foreach { gp $_.PSPath }'
# ' | ? { $_ -match "Certbot" }'
# ' | select UninstallString)'
# '.UninstallString', capture_stdout=True)
# subprocess.check_call([uninstall_path, '/S'])
pass
def _ps(powershell_str, capture_stdout=False):
fn = subprocess.check_output if capture_stdout else subprocess.check_call
return fn(['powershell.exe', '-c', powershell_str], universal_newlines=True)